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The  two-level  maintenance  concept  for  field  support  is 
being  considered  for  many  new  avionics  systems.  Decisions 
must  be  made  early  in  the  development  program  as  to  whxch 
maintenance  concept  to  pursue.  This  thesis  develops  a 
technique  to  quantify  the  operational  suitability,  in  terms 
of  its  stated  operational  requirements,  of  a  system 
supported  With  a  two-level  maintenance  concept. 
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Abs 

The  purpose  cf  this  study  was  to  develop  a  technique 
for  evaluating  operationai  suitability  of  new  avionics 
systems  with  a  two-level  maintenance  support  system.  The 
technique  was  primarily  needed  early  in  the  design  phases  of 
development  programs.  The  focus  would  be  on  could  a  system 
meet  its  operationai  requirements  as  set  down  in  program 
documentation  such  as  the  Systems  Operational  Requirements 
Document  (SORD)  with  a  two-level  maintenance  concept. 

The  study  had  two  objectives:  develop  the  technique 
and  demonstrate  it  on  a  real  system.  The  AN/ALE-47 
Countermeasures  Dispenser  System  was  used  to  demonstrate  the 
technique.  A  SLAM  II  simulation  model  was  developed  which 
used  operational  requirements  and  three  system  factors  ar 
input  parameters.  The  system  factors  were  reliability, 
testability,  and  the  initial  spares  purchase. 

Probability  distributions  were  developed  for  the  mean 
and  minimum  monthly  theater  spares  level.  The  average  drop 
in  monthly  spares  level  due  to  the  stated  war  surge  was 
determined.  The  probability  of  having  enough  spares  to 
handle  the  war  surge  drop  was  determined.  This  allowed 
quantification  of  the  operational  suitability  of  the  system 
with  the  two-level  concept. 
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An  additional  analysis  determined  the  general  effects 
of  each  of  the  system  factors  on  the  monthly  theater  spares 
level  through  the  use  of  a  2*^  Factorial  Design.  This 
information  aids  the  user  in  determining  which  factors 
should  receive  attention  to  improve  the  operational 
suitability  of  the  system  with  a  two-level  maintenance 
concept . 
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AN  OPERATIONAL  SUITABILITY  EVALUATION  TECHNIQUE 


FOR  AVIONICS  SUPPORTED  WITH  TWO-LEVEL  MAINTENANCE 

CONCEPTS 


I.  Introduction 


Background 

Avionics  f the  electronics  onboard  aircraft'  have  become 
so  complex  and  numerous  that  a  significant  portion  of  the 
Air  Force  maintenance  structure  exists  simply  to  support 
them.  The  Air  Force  implements  its  maintenance  structure  in 
three  levels  consisting  of  organisational  maintenance  at  the 
flight  line,  intermediate  maintenance  at  a  base  level  shop, 
and  depot  maintenance  at  a  central  location.  From  a 
wai’fighting  capability  perspective,  there  are  several 
disadvantages  with  the  three-level  maintenance  structure. 

It  generates  a  large  mar  power  requirement  and  is  vulnerable 
during  a  war  because  of  the  required  facilities  and 
manpower.  The  maintenance  structure  must  be  protected  and 
supported  Just  as  the  aircraft,  weapons,  and  supplies  are. 
Mobility  is  an  important  issue  in  today’s  weapon  systems 
since  the  intermediate  maintenance  support  must  be  deployed 
to  the  conflict  area  with  the  weapons.  In  short,  the  three- 
level  concept  is  large,  bulky,  and  ties  up  airlift  resources 
to  move  it  to  the  new  location,  thus  diminishing  warfightir.g 


I 


capability.  The  Air  Force  wanted  to  improve  warfighting 
capability,  and  in  1984  came  out  with  the  ' R  &  M  2000" 
initiative. 

The  R  &  M  2000  initiative  recognizes  the  importance  of 
reliability  and  maintainability  of  weapon  systems  to 
warfighting  capability.  The  initiative  requires  quantified 
goals  of  new  systems  for: 

A.  Increasing  warfighting  capability. 

B.  Increasing  survivability  of  the  support 
infrastructure . 

C.  Improving  mobility. 

D.  Lowering  manpower  requirements. 

E.  Decreasing  costs. 

These  goals  are  all  supported  by  increasing  system 
reliability,  improving  fault  detection  and  isolation,  and 
reducing  the  need  for  ma Intenance  (11:1). 

In  response  to  R  &  M  2000,  an  alternative  to  the  three- 
level  concept,  the  two-level  concept  is  being  considered  for 
many  avionics  systems.  This  concept  eliminates  the 
intermediate  level  of  repair  at  the  base  level  and 
alleviates  some  of  the  problems  associated  with  the  three 
level  concept.  Avionics  which  fail  on  the  aircraft  are 
removed  and  replaced  with  spares.  The  failed  units  are  then 
shipped  directly  to  the  depot  for  repair.  The  two-level 
concept  could  be  a  way  to  streamline  the  maintenance  support 
structure,  providing  the  system  is  designed  for  support  in 
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this  manner.  In  theory,  less  manpower  and  support  equipment 
are  needed  at  the  base  level  to  support  operations.  If  so, 
a  more  efficient,  less  vulnerable,  and  more  easily  deployed 
weapon  system  would  result.  Such  a  system  would  support  the 
R  &  M  2000  goals  and  increase  th^  Air  Force’s  warfighting 
capab i 1 i ty . 

General  Issue 

There  are  many  design  decisions  which  affect  whether  a 
system  can  support  flying  operations  with  a  two-level 
maintenance  concept.  Design  decisions  involve  the  system 
reliability,  maintainability,  and  testability.  These 
decisions  generally  occur  well  before  the  system  design  is 
finalized.  Air  Force  personnel  and  defense  contractors 
would  like  to  know  the  design  factor  levels  the  system  must 
achieve  to  support  flying  operations  with  a  two  level 
maintenance  concept.  No  means  are  currently  available  for 
arriving  at  these  levels. 

System  program  office  engineers  need  a  tool  which  will 
allow  rapid  and  accurate  quantification  of  the  effects 
’.’arlo"S  levels  of  reliability,  maintainability,  and 
testability  will  have  on  the  system's  capability  to  support 

^  flying  operations.  Such  a  tool  would  link  operational 

requirements  and  system  development  parameters,  clearly 

1 

showing  the  effects  of  the  system’s  design  parameters  on 
operations.  Since  it  is  difficult  to  significantly  alter 


3 


reliability,  ma i n ta i nab i 1 i ty ,  or  testability  after  the 
design  is  finalized,  the  tool  must  be  utilized  early  in  the 
system  design.  However,  early  in  the  program,  exact 
information  is  not  always  available  so  the  model  should 
utilize  the  stated  operational  requirements  from  the 
applicable  System  Operational  Requirements  Document.  The 
using  command  provides  this  document  to  all  programs  in  the 
early  phases.  A  tool  available  early  in  the  program  would 
allow  what  if  analyses  to  be  performed  within  program  and 
operational  constraints. 

Another  important  decision  in  operational  suitability 
is  determining  the  number  of  spares  to  purchase  to  support 
the  system.  This  decision  is  mostly  related  to  program 
funding,  but  can  be  more  easily  varied  system  design 
parameters.  Any  tool  developed  should  aid  the  program 
office  in  determining  the  number  of  spares  needed  to  support 
the  system  with  a  two-level  concept. 

Cost  models  exist  which  develop  the  costs  associated 
with  either  a  two-level  or  a  three-level  maintenance 
concept.  Cost  models  generally  only  address  the  economy  and 
not  the  effectiveness  of  the  maintenance  concept.  The 
lowest  cost  solution  may  not  alv;ays  be  the  b^st  solution  in 
terms  of  capability  to  support  a  system  in  all  scenarios. 
Maintenance  concept  effectiveness  (or  operational 
suitability)  and  economy  should  be  considered  when  making  a 
system  support  concept  decision.  Air  Force  System  Program 
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Office  personnel  are  responsible  for-  making  a  decision  based 
on  the  economy  and  the  effectiveness  of  the  system  and  its 
maintenance  concept. 

Problem  Statement 

SPO  personnel  need  the  capability  to  assess  operational 
suitability  of  proposed  maintenance  concepts  based  on  system 
design  parameters,  operational  requirements,  and  projected 
spares  purchases. 

Research  Objective 

The  objective  of  this  research  is  to  develop  and 
demonstrate  a  technique  to  evaluate  operational  suitability 
of  proposed  maintenance  concepts  for  a  new  weapon  system. 

Sub-Ob j  ec 1 1 ve 

This  technique  will  be  demonstrated  on  a  new  weapon 
system  by  providing  an  analysis  of  a  proposed  maintenance 
concept  to  a  System  Program  Office.  The  system  selected  for 
demonstration  purposes  is  the  ALE-47  Countermeasures 
Dispenser  System.  The  ALE-47  Program  Office  must  decide  on 
a  maintenance  concept.  The  office  would  like  to  evaluate 
the  operational  suitability  of  the  ALE-47  with  alternative 
maintenance  concepts  before  making  the  decision.  The  office 
believes  this  is  an  important  aspect  which  must  be 
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considered  in  the  decision.  This  will  demonstrate  the 
technique  in  the  system  program  office  environment. 

Research  Questions 

The  following  questions  focus  the  research  effort  on 
the  information  needed  to  evaluate  a  system’s  operational 
suitability  supported  by  a  two-level  maintenance  concept: 

1.  Can  a  probability  distribution  be  developed  for  the 
average  monthly  spares  level  at  each  operating  location? 

2.  Can  a  probability  distribution  be  developed  for  the 
minimum  monthly  spares  level  at  each  operating  location? 

3.  What  is  the  effect  of  wartime  surge  sortie 
generation  rates  on  the  system? 

4.  Can  the  influences  of  the  design  factors  on  the 
system’s  operational  capability  be  quantified? 

Limitation  of  Scope 

This  research  will  not  attempt  to  address  the  issue  of 
cost  of  the  two-level  maintenance  concept.  Life  Cycle  Cost 
models  exist  and  are  available  to  the  ALE-47  Program  Office 
which  can  calculate  costs  for  the  different  maintenance 
systems.  This  research  will  focus  on  determining  whether 
the  subsystem  as  designed  can  meet  the  operational 
requirements  placed  on  the  overall  system. 
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Definition  of  Ter ms 

Unfamiliar  terms  are  defined  below.  These  are  common 
operational  terms  used  in  the  Air  Force. 

Avionics  -  electronics  carried  on  aircraft.  The 
avionics  discussed  in  the  research  will  be  the  ALE-47 
Countermeasures  Dispenser  System. 

Break  Rate  -  an  operational  term  reflecting  the 
failures  which  require  immediate  maintenance  to  restore. the 
system  to  mission  capable  status.  Break  Rate  is  the  failure 
rate  of  the  system  derated  to  a  sortie  basis.  It  is  equal 
to  one  minus  the  mission  reliability  for  the  system  (10: 14) . 

Sortie  Generation  Rate  (SQR)  -  the  average  number  of 
sorties  produced  per  aircraft  in  a  defined  operating  day 
( 10 : 38)  . 
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II.  Literature  Review 


Introduction 

The  literature  review  focuses  on  general  topics 
relating  to  the  research  task.  The  ALE-47  and  its 
operational  requirements  are  described  and  discussed.  Where 
and  how  the  ALE-47  fits  in  with  F-16  initiatives  toward  two- 
level  maintenance  is  discussed.  The  two-level  maintenance 
concept  is  described  and  contrasted  with  the  three-level 
maintenance  concept  which  is  widely  used  for  avionics  today. 
The  use  of  simulation  in  research  such  as  this  is  also 
discussed.  Finally,  alternative  models  which  could  be 
utilized  in  the  research  are  discussed. 

AN/ALE-47  Countermeasures  Dispenser  System 

The  ALE-47  Countermeasures  Dispenser  System  is  a 
replacement  system  for  current  operational  dispenser 
systems.  The  program  is  a  response  to  degraded  aircraft 
survivability  caused  by  more  sophisticated  threats.  A 
computer  controlled,  reprogrammable  dispenser  with  the 
capability  to  automatically  or  manually  eject  chaff,  flares 
and  expendable  Jammers  was  needed.  It  was  originally 
conceived  to  replace  the  Air  Force’s  AN/ALE-40,  but  the  Navy 
also  needed  a  replacement  for  its  AN/ALE-39.  The 
Undersecretary  of  Defense  for  Acquisition  issued  a  directive 
to  combine  the  two  service’s  replacement  efforts  into  one 
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program  in  November  1986.  This  single,  Joint  program  would 
satisfy  the  requirements  of  both  services.  In  addition,  the 
Army  needed  a  replacement  for  its  M-130  dispenser  system  so 
its  requirements  were  added  to  the  program.  The  ALE-47  will 
be  a  common  countermeasures  dispenser  system  for  ail 
services  and  should  reduce  operation  and  support  costs  to 
the  Department  of  Defense  (2:4). 

Since  the  ALE-47  will  be  common  to  both  the  Air  Force 
and  Navy,  the  line  replaceable  units  will  be  essentially  the 
same  but  arranged  in  different  configurations.  One  other 
difference  will  be  present.  The  Navy  uses  round  expendables 
and  the  Air  Force  uses  square  expendables.  This  is  handled 
by  a  replaceable  magazine  insert  which  conforms  to  the 
service’s  expendables  (4:61).  The  Air  Force  version  of  the 
ALE-47  consists  of  7  Line  Replaceable  Units:  1  Cockpit 

Control  Unit;  1  Programmer;  1  Safety  Switch;  2 
Sequencers;  2  Dispenser  Assemblies  (2:4) .  Different 
magazine  assemblies  can  be  inserted  into  the  dispenser 
assemblies  allowing  the  loading  of  different  sizes  and  types 
of  expendables.  The  ALE-47  will  be  able  to  sense  the 
different  loads  carried  onboard  and  upon  receiving  threat 
information  from  other  systems,  can  dispense  the  proper 
countermeasure.  The  system  may  also  be  activated  manually 
(2:4-5)  . 

The  ALE-47  is  being  developed  by  Tracer  Incorporated  on 
a  four-year  full  scale  development  contract  from  the  Air 
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Force.  Tracor  Inc.  is  based  in  Austin,  Texas  and  is  an 


industry  leader  in  passive  countermeasures.  The  contract 
price  is  15.6  million  dollars  and  includes  five  production 
options  valued  at  77  million  dollars  (5:105). 

Table  I  outlines  the  operational  requirements  for  the 
ALE-47  as  stated  in  the  system  specification. 


Table  I.  Specification  Parameters 

Parameter  Value 

a.  Host  aircraft  SGR 

(sorties/aircraft/d ay) 


Wartime  (18  hr  day) 

Day  0  to  Day  6  3.1 

Day  7  to  Day  29  2.0 

Day  30  and  on  1.5 

Peacetime  (12  hr  day)  1.2 

b.  Sortie  Duration  (hours) 

Wartime  2 . 0 

Peace t i me  i  .  5 

c.  False  Removal  Hate  (percent)  1.0 

d.  Breakrate  (percent) 

Wartime  0.195 

Peacetime  0.146 

e.  Mean  Repair  Time  (hours) 

On  equipment  0.333 


(  1  :  150-151) 


The  maintenance  support  concept  has  not  been  chosen  at 
this  time.  The  three  and  two-level  concepts  are  being 
considered.  The  program  office  believes  that  the  ALE-47 
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could  be  supported  with  either  but  would  prefer  to  recommend 
the  most  effective  and  economical  concept. 

Current  Maintenance  Operations 

The  Air  Force  currently  uses  a  three  level  maintenance 
concept  for  avionics.  This  concept  is  efficient, 
maintaining  peacetime  readiness  at  an  acceptable  level  while 
providing  scarce  part  availability  during  wartime  surge 
modes.  It  also  gives  personnel  in  the  maintenance  system 
the  task  of  repairing  items  at  their  particular  level.  This 
allows  each  person  to  identify  with  his  part  in  the 
maintenance  process  (3:72). 

There  are  some  drawbacks  to  three  levels  of 
maintenance.  It  creates  a  large  and  cumbersome  system  with 
large  manpower  requirements.  Maintenance  manpower 
requirements  can  be  as  high  as  1,800  slots  for  a  72  aircraft 
fighter  wing  (6:3) .  This  number  includes  people  at  all 
levels  of  maintenance.  Large  numbers  of  personnel  creates  a 
large  and  vulnerable  main  operating  base  which  must  be 
protected  just  as  the  weapons  are.  This  can  tie  up  valuable 
wartime  assets  protecting  the  air  base  instead  of  waging 
war.  To  decrease  the  vulnerability,  the  Air  Force  plans  to 
disperse  fighter  wings  into  small  units,  making  it  harder  to 
find  the  aircraft  (6:3) .  The  drawback  with  dispersal  is 
that  under  a  three  level  maintenance  policy,  maintenance 
manpower  requirements  would  increase  dramatically.  Under 


today’s  policy,  deploying  a  squadron  of  24  aircraft  (based 
on  F-15  and  F-16  fighters)  requires  about  600  maintenance 
men  per  squadron  (6:3) .  Simulation  of  F-16  operations  have 
shown  that  deploying  a  wing  of  72  into  18  groups  of  C 
aircraft  each  could  almost  double  the  manpower  requirements 
because  of  the  different,  unique  skills  of  the  maintainers 
(6:3).  If  a  two  level  concept  could  be  i mp lemented  the 
maintenance  personnel  requirements  at  the  intermediate  level 
could  be  eliminated  which  would  drastically  reduce  the 
manpower  requirements  for  a  deployment  or  a  dispersal. 

Two-level  Maintenance  Operations 

The  goal  of  using  a  two  level  maintenance  concept  is  to 
improve  unit  mobility  and  flexibility  by  reducing  manpower 
requirements.  Failed  units  would  be  repairf  at  the 
aircraft  on  the  flightline  or  removed  and  sent  to  the  depot 
for  repair.  This  should  simplify  maintenance  at  the 
organizational  level  since  only  limited  repair  work  could  be 
performed  on  the  aircraft.  It  should  also  reduce  the 
airlift  requirements  during  a  deployment  because  no 
intermediate  level  support  equipment  will  have  to  be  brought 
along  for  avionics  repair.  The  maintenance  structure  is 
smaller,  less  vulnerable,  and  can  be  deployed  more  rapidly 
(3:72) . 

The  drawback  with  two  level  maintenance  is  the  greater 
dependency  on  the  depot  to  quickly  repair  the  failed  items 
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and  get  them  back  to  the  operational  bases.  Airlift  will  be 
tied  up  assuring  failed  items  get  to  the  depot  and  repaired 
items  get  back  to  the  bases.  Inventory  control  will  have  to 
be  tighter.  The  system  also  depends  on  high  reliability  of 
the  items  removed  from  the  aircraft  to  cut  down  on  the 
number  of  times  they  are  shipped  to  the  depot  for  repair 
(3:72).  Another  dependency  is  on  the  ability  to  properly 
identify  a  failed  item.  Once  an  item  is  removed  it  is  sent 
all  the  way  to  the  depot  before  it  is  checked  again  to  see 
if  it  really  is  failed.  If  this  percentage  is  too  high, 
sortie  generation  capability  will  suffer  over  time. 

One  final  drawback  is  the  loss  of  the  flightline 
maintainer's  identification  with  real  repair  work.  The 
flightline  maintainer  will  simply  be  removing  failed  items 
and  sending  them  to  the  depot  for  repair.  The  feeling  of 
fixing  the  aircraft  will  not  be  as  strong  as  when  actual 
repair  work  was  performed  on  the  system.  This  effect  on 
personnel  will  have  to  be  accounted  for  when  considering  a 
two-level  concept  (3:72' .  The  F-16  Program  Office  is  ■ 
studying  the  two-level  concept  for  selected  avionics. 

F-16  Two-Level  Initiatives 

The  F-16  System  Program  Office  has  an  ongoing  Support 
System  Alternatives  Study  with  General  Dynamics.  The  study 
focuses  on  the  Avionics  Intermediate  Shop  (AIS)  of  the 
The  objective  of  the  study  is  to  minimize  the  AIS  through 
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reliability  improvement  and  reduction  of  "cannot  duplicates" 
in  the  avionics  which  it  supports  (12:4) .  Improving 
reliability  could  allow  some  systems  currently  supported  on 
the  AIS  to  be  switched  to  a  two-level  maintenance  concept. 
Reliability  goals  which  allow  switching  to  a  two-level 
concept  must  be  determined.  The  goals  are  determined  by 
life  cycle  repair  costs  for  repairing  the  avionics  at  the 
various  levels  of  repair.  The  lowest  overall  cost  is 
assumed  to  the  opti mum  level  (12:11-12).  Cannot  Duplicates 
are  defined  as  items  which  confirm  as  failed  on  the  aircraft 
but  upon  later  testing  off  the  aircraft  check  as  serviceable 
(12:21)  .  These  become  even  more  important  in  a  two-level 
concept  because  the  off  aircraft  testing  is  done  at  the 
depot  instead  of  the  AIS. 

The  ultimate  objective  of  the  this  study  is  to  support 
the  RScM  2000  goals.  Cost,  manpower,  and  mobility 
requirements  will  be  reduced  by  having  a  smaller  AIS. 
Survivability  and  combat  capability  will  be  improved  by 
avionics  improvements  and  a  smaller  AIS  (12:7)  . 

The  ALE-47  would  be  supported  on  the  AIS  if  a  three- 
level  concept  13  decided  upon.  A  two -level  concept  would  be 
more  in  line  with  the  F-16  initiatives  and  help  to  reduce 
the  AIS  size.  This  should  only  be  pursued  if  eifective 
support  can  be  achieved  without  reducing  warfighting 
capability.  Some  modeling  technique  should  be  used  to 
determine  if  capability  will  be  affected. 


Alternative  Modeling  Techniques 


There  are  several  alternative  methods  to  evaluate  a 
two-level  maintenance  system.  The  system  could  be  fully 
implemented  and  then  evaluated.  This  is  not  an  acceptable 
alternative  because  of  the  risks  involved  in  implementing  a 
system  before  its  capability  to  support  the  weapon  system 
has  been  demonstrated. 

Another  alternative  is  the  development  of  a 
deterministic  inventory  model  which  accurately  models  the 
system  since  the  maintenance  system  is  basically  an 
inventory  system.  The  are  two  main  assumptions  associated 
with  deterministic  inventory  models.  Demand  and  lead  time 
must  be  constant  ( 26 : 632-683)  .  Such  a  model  would  not  be 
accurate  since  neither  hold  true  for  a  maintenance  system. 

A  stochastic  process,  such  as  a  Markov  Chain,  could 
possibly  be  used  to  model  the  system.  A  system 
characteristic  which  varies  with  time  is  observed  at 
discrete  points  in  time  in  a  Markov  Chain  (26:738-739) . 
Markov  Chains  require  the  various  states  of  the  model  to  be 
independent  of  time  (26:738-739) .  A  Markov  Chain  could 
possibly  be  used  to  model  the  system  if  it  remained 
essentially  stationary  such  as  in  a  peacetime  environments. 
The  dynamic  behavior  the  system  exhibits  during  surges  could 
not  properly  be  handled  by  a  Markov  Chain. 
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A  simulation  model  could  be  used  to  evaluate  the  two- 
level  maintenance  system.  Using  simulation  overcomes  the 
problem  of  conducting  experiments  on  the  real  system. 
Experimenting  with  a  real  system  takes  more  time,  cannot,  be 
replicated,  costs  more,  and  does  not  allow  the  extreme 
conditions  to  be  analyzed  safely  (20r7) .  Analytical  models 
of  real  systems  can  be  developed  but  the  real  system  is 
often  too  complex  for  the  analytical  model  to  accurately 
predict  behavior.  Simulation  allows  a  model  to  be  evaluated 
over  time  to  estimate  the  model’s  characteristics  through 
the  use  of  computers  (18:1)  .  This  allows  analysis  of  a 
proposed  system  without  actually  having  to  build  it  before 
determining  its  characteristics.  There  exists  a 
misconception  that  simulation  is  simply  writing  a  program 
and  getting  results.  The  methodology  of  the  simulation 
experiment  is  just  as  important  as  the  building  of  the  model 
itself  (18:2)  . 

There  are  several  advantanges  of  simulation. 

Analytical  models  often  cannot  accurately  model  real 
systems.  Systems  can  be  analyzed  under  different  operating 
conditions.  Alternative  designs  can  be  compared  without 
having  to  build  them.  Control  of  the  experiment  can  be 
maintained  when  using  simulation  unlike  performing 
experiments  on  real  world  systems  and  simulation  allows  the 
system  to  be  studied  for  long  periods  of  time  without 
actually  having  to  operate  it  for  that  long  (18:8)  . 


There  are  also  disadvantages  associated  with  using 

simulation.  Simulation  is  not  cheap  and  is  not  always  a 

time  saving  technique.  Simulation  is  not  reality.  Only 

estimates  of  reality  can  be  obtained  from  simulation.  Often 

more  faith  is  put  in  simulation  results  than  is  deserved 

because  of  the  enormous  amounts  of  data  generated  (18:8-9) . 

The  advantages  and  disadvantages  must  be  kept  in  mind 

when  deciding  to  conduct  a  simulation  study.  In  the  case  of 

the  ALE-47,  no  real  system  exists  right  now  but  a  policy 

must  be  decided.  An  estimate  of  the  system's  behavior  over 

its  operating  life  before  actually  fielding  it  would  be  very 

beneficial  to  the  policy  decision  process.  Since 

maintenance  systems  are  dynamic  and  must  be  evaluated  under 

several  sets  of  conditions,  simulation  appears  to  be  an 

acceptable  method  of  evaluation. 

Once  the  decision  has  been  made  to  use  simulation,  the 

question  arises  whether  to  use  an  existing  model  or  to  build 

a  new  one.  Dyna-METRIC  will  be  examined  as  a  possible 

candidate  for  the  research. 

Dyna-METRIC  was  developed  by  the  Rand  Corporation  to 

provide  information  to  improve  wartime  logistics  support. 

The  Dyna-METRIC  model  was  developed  to  study  and 
predict  the  readiness  of  groups  of  aircraft 
squadrons  as  determined  by  a  major  subset  of 
logistics  resources,  namely,  those  associated  with 
component  repair  and  resupply  (15:2) . 

Dyna-METRIC  is  a  synthesis  of  analytical  and  mathematical 

models  with  simulation  (15:4-5) .  This  allows  it  to  handle 
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the  dynamic  behavior  exhibited  by  the  system  when  sortie 
rates  vary  (15:4) .  It  breaks  the  maintenance  system  down 
into  various  sections  such  as  sortie  generation,  supply, 
shipping,  and  repair.  These  sections  are  designated  as 
■pipelines'  which  must  be  filled  with  enough  spares  to  allow 
the  system  to  operate  properly  (15:3) .  Calculations  are 
made  to  determine  the  number  of  spares  needed  to  fill  the 
various  pipelines  (15:30) .  This  measure  can  be  used  to 
develop  the  spares  requirements  for  the  system.  The 
probability  of  meeting  the  required  missions  is  also 
calculated  to  provide  a  measure  of  operational  effectiveness 
(15:37)  . 

"Version  4  of  Dyna-METRIC  was  developed  expressly  to 
assess  worldwide  logistic  support  of  aircraft  components, 
including  the  depot-theater  interactions'  (16:1).  It 
relates  logistics  resources  and  policies  to  wartime 
readiness.  It  provides  a  measure  of  operational  performance 
under  the  effects  of  wartime  and  repair  constraints.  It 
also  provides  problem  detection  and  spares  requirements.  It 
is  designed  to  provide  three  levels  of  maintenance  down  to 
three  levels  of  system  repair.  The  three  levels  of  repair 
are  the  Line  Replaceable  Unit,  the  Shop  Replaceable  Unit, 
and  the  piece  part  level.  An  aircraft  is  modeled  as  a 
collection  of  LRU’s  which  must  all  be  working  for  an 
aircraft  to  complete  its  sorties. 
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There  are  several  assumptions  made  in  Dyna-METRIC  which 


are  necessary  for  it  to  operate  correctly.  Repair  is 
unconstrained.  Aircraft  availability  and  sortie 
generation  capability  are  the  operational  performance 
parameters  provided  as  output.  Lists  of  problem  components, 
a  report  of  the  depot  workload,  and  recommended  spares 
levels  are  also  provided  (16:8-10). 

Dyna-METRIC  does  not  appear  to  be  suited  to  this 
research  due  to  the  assumptions  which  are  made  in  it  for 
operation.  It  requires  much  greater  input  information  than 
is  available  on  the  ALE-47.  It  also  doc.a  not  provide  a  good 
measure  of  the  dynamics  of  th^  system  which  is  necessary  for 
this  study.  The  best  alternative  would  be  to  develop  a  new 
model  optimizes  for  the  research  task  of  analyzing  a  single 
system  which  accepts  the  standard  System  Operational 
Requirements  Document  data  as  it  is  provided. 

Simulation  Languages 

Once  the  decision  has  been  made  to  develop  a  new  model, 
a  simulation  language  must  be  selected.  The  availability  of 
the  language,  the  type  of  problem,  the  amount  of  time 
available,  and  the  programming  effort  are  all  important 
considerations  when  selecting  a  language.  Also  important  is 
the  computer  languages  which  are  known  by  the  researcher 
(18:133).  SLAM  II  is  available  to  AFIT  and  the  ALE-47 
system  program  office  on  mainframe  computers.  Its 
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capabilities  are  well  suited  for  the  research  task.  Because 
of  these  factors,  SLAM  II  on  the  mainframe  computer  was 
chosen  as  the  simulation  language  for  the  research  effort. 

SLAM  II  is  the  Simulation  Language  for  Alternative 
Modeling  (21:2) .  It  supports  modeling  of  systems  from  a 
variety  of  points  of  view.  It  is  capable  of  discrete  event 
modeling  with  an  event  orientation,  a  process  orientation, 
or  both.  It  is  also  capable  of  continuous  event  modeling. 
SLAM  II  allows  a  pictorial  representation  of  the  system  to 
be  built  using  nodes  and  branches  which  is  important  for 
documenting  the  model  to  the  ALE-47  Program  Office.  The 
nodes  and  branches  are  symbols  which  model  elements  in  a 
process.  Developing  a  SLAM  II  model  requires  combining 
these  symbols  into  a  network  which  represents  the  process 
being  modeled.  Entities  then  flow  through  the  network 
during  operation.  The  network  can  be  built  to  the  level  of 
detail  necessary  for  the  simulation  study.  After  the 
network  is  built,  it  is  translated  for  computer  execution 
(21:62-63) .  Output  can  be  transported  to  personal  computer 
software  packages  for  analysis. 

Summary 

This  chapter  has  provided  a  review  of  the  current 
literature  on  general  subjects  associated  with  this 
research.  It  presented  information  on  the  general  system  to 
be  modeled,  the  two-level  maintenance  system,  and  on  the 
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specific  system,  the  ALE-47.  Alternative  modeling 
techniques  were  examined  and  a  method  selected. 
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Introduction 

An  examination  of  the  operational  suitability  of  the 
ALE-47  with  a  two-level  maintenance  concept  is  required  to 
fulfill  the  research  objectives.  Since  the  ALE-47  does  not 
physically  exist  today,  a  model  of  the  two-level  maintenance 
support  structure  is  needed  for  evaluation  purposes.  The 
previous  chapter  discussed  the  two-level  maintenance  concept 
and  its  assumptions.  Examination  of  alternative  models 
revealed  that  none  are  particularly  well  suited  for  the 
research  task.  It  was  decided  to  perform  a  simulation  study 
with  a  newly  developed  SLAM  II  model.  The  methodology 
defines  the  overall  approach  for  model  development  and  the 
experiments  conducted  with  the  model.  Law  and  Kelton 
outline  a  ten-step  approach  to  conducting  a  simulation  study 
which  is  applicable  for  this  research  effort  (18-43-46). 
Other  authors,  such  as  Kleljnen  (17) ,  Pidd  (20) ,  and 
Pritsker  (21)  provide  similar  outlines.  The  steps  are  as 
f ol lows ; 

1.  Formulate  the  problem  and  plan  the  study. 

2.  Collect  data  and  define  a  model. 

3.  Check  for  validity  of  the  model. 

4.  Construct  a  computer  program  and  verify. 

5.  Make  pilot  runs. 

6.  Check  for  validity  of  output. 
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7.  Design  the  experiments. 

8.  Make  production  runs. 

9.  Analyze  output  data. 

10.  Document  and  implement  results. 

These  general  tasks  must  be  accomplished  to  conduct  a 
simulation  study.  This  chapter  addresses  steps  1  through  8. 
Chapters  IV  and  V  address  steps  9  and  10  respectively. 

Problem  definition 

An  analysis  of  the  operational  suitability  of  the  ALE- 
47  Countermeasures  Dispenser  System  with  the  two-level 
maintenance  concept  is  needed  for  decision  purposes.  There 
are  two  main  reasons  to  use  simulation  in  a  study  such  as 
this.  The  comparison  of  model  responses  under  different 
conditions  is  the  first  and  the  second  is  to  investigate  the 
relationships  between  the  independent  factors  and  the 
model’s  response  (17:65) .  In  the  case  of  the  ALE-47,  both 
are  reasons  for  conducting  the  simulation  study.  Combat 
capability  in  peacetime  and  wartime  under  various 
constraints  needs  to  be  quantified.  The  weights  of  the 
factors  in  relation  to  the  response  is  important  for 
determining  possible  tradeoffs  in  case  the  system  does  not 
achieve  its  warranted  requirements  or  the  amount  of  spares 
purchased  is  different  from  the  anticipated. 

The  ALE-47  SPO  must  make  a  decision  on  which 
maintenance  concept  to  pursue  for  the  system.  Operational 


suitability  is  only  one  of  several  criteria  to  be  considered 
when  making  such  a  decision.  This  study  will  evaluate  the 
operational  suitability  of  the  ALE-47  with  the  two-level 
maintenance  concept.  The  results  will  be  used  to  determine 
how  effectively  the  concept  meets  system  operational  goals. 
The  results  of  this  experiment  will  provide  decision  support 
information  in  the  area  of  operational  suitability  to  be 
considered  along  with  other  data  when  making  the  maintenance 
concept  decision. 

Base  spares  level  is  the  measure  of  operational 
suitability  chosen  for  this  research.  It  is  defined  as  the 
number  of  spares  available  on  a  daily  basis  at  each  base. 
Fighter  wings  must  have  minimum  levels  of  spares  to  be 
operationally  effective.  If  no  spare  is  available,  sorties 
must  be  flown  without  capability  of  the  subsystem.  Sorties 
will  not  be  flown  if  subsystems  on  the  mission  essential 
subsystem  list  (MESL)  are  not  operational.  There  are  two 
mission  essential  subsystem  lists,  one  for  peacetime  and  one 
for  wartime.  Longer  duty  days  and  higher  sortie  rates  are 
required  in  time  of  war.  The  ALE-47  would  most  likely  not 
be  on  the  MESL  in  peacetime  since  it  is  seldom  exercised 
except  for  training  purposes.  However,  it  may  be  present  on 
the  list  in  time  of  war  since  proper  operation  of  the  ALE-47 
can  greatly  enhance  aircraft  survivability.  Its  presence  on 
the  MESL  has  not  been  determined  at  this  time.  System 
utilization  would  have  been  implemented  as  a  percentage  of 


sorties  flown  for  peacetime  and  wartime  but  values  could  not 
be  quantified.  It  is  a  program  issue  that  has  yet  to  be 
settled.  System  utilization  was  assumed  to  be  100  percent 
since  a  check  of  the  ALE-47  would  be  made  before  each  sortie 
and  if  a  failure  was  detected,  maintenance  would  be 
schedul ed . 

The  spares  level,  as  defined  above,  should  be  a 
response  to  several  factors  ranging  from  system  design  to 
Air  Force  policies.  Three  independent  factors  have  been 
identified  for  this  study.  Reliability,  testability,  and 
spares  purchase  size  of  the  ALE-47  are  the  three  factors. 

Reliability  is  implemented  in  terms  of  the  system 
breakrate .  Breakrate ,  as  defined  earlier,  is  a  percentage 
of  the  time  a  system  returns  from  a  mission  with  a  failure. 
The  percentage  is  based  on  the  exponential  failure  rate  of 
the  system  for  a  set  mission  duration.  The  percentage  must 
be  recalculated  for  a  change  in  mission  duration. 

Testability  is  implemented  as  a  of  percentage  of  false 
removals.  False  removals  occur  when  a  unit,  thought  to  be 
failed,  is  removed  from  the  aircraft  and  sent  through  the 
repair  process.  Upon  later  inspection,  the  unit  turns  out 
not  to  be  failed.  This  is  an  additional  percentage  of  the 
breakrate  on  a  sortie  basis. 

Spares  purchase  size  is  given  in  terms  of  a  percentage 
of  the  production  buy.  The  program  office  provided  20 
percent  as  a  planning  figure  for  the  purpose  of  this  study. 
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Data  Collection 


Gathering  sufficient  information  about  the  proposed 
maintenance  system  to  allow  simulation  model  development  is 
an  important  step  in  the  process.  A  list  of  the  data 
necessary  to  develop  the  model  follows. 

1.  Number  of  bases,  location,  and  number  of  aircraft 
assigned  for  both  peacetime  and  wartime. 

2.  Required  system  operational  requirements. 

3.  Shipping  times  for  failed  items  from  each  base  to 
the  depot . 

4.  Incoming  inspection  rates  for  items  which  pass 
operational  status  tests. 

5.  Depot  maintenance  times  for  the  repair  tasks. 

6.  Quality  assurance  reject  rates  for  repaired  items 
(those  which  are  still  not  operational ) . 

7.  Shipping  times  to  bases  for  repaired  items. 

These  Inputs  will  be  gathered  by  using  the  ALE-47  Use 

Study  (2) ,  the  ALE-47  Specification  (1) ,  and  discussion  with 
program  personnel.  The  logistics  manager  was  consulted  as 
to  the  correctness  of  the  maintenance  system  model  and  to 
provide  any  additional  logistics  data  needed.  System 
definition  was  provided  by  the  system  engineer.  A  walk 
through  of  the  model  was  accomplished  with  the  logistics 
manager  and  the  reliability  engineer  to  determine  the 
validity  of  the  model. 
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The  simulation  model  is  divided  into  three  modules,  the 
sortie  generation  module,  the  base  supply  module,  and  the 
deoot  module.  The  flying  operations  and  maintenance 
procedures  at  each  base  are  modeled  in  the  sortie  generation 
and  supply  modules.  The  depot  repair  process  is  modeled  in 
the  depot  module.  The  two-level  model  contains  the  sortie 
generation,  base  supply,  and  depot  modules.  Discussion  of 
the  modules  follows. 

The  sortie  generation  module  models  the  sortie 
generation  capability  in  each  theater  where  the  system  will 
be  used.  The  system’s  breakrate  is  on  a  per  sortie  basis 
and  will  be  used  to  generate  the  system  failures.  Spare 
LRU’s  are  maintained  for  each  theater.  These  spares  will  be 
used  to  replace  units  on  the  aircraft  as  they  fail.  Enough 
spares  must  be  available  to  meet  the  sortie  goals. 

The  supply  module  will  collect  the  failed  LRU’s  and 
prepare  them  for  shipping  to  the  depot.  The  units  will 
await  weekly  shipping  in  this  module.  Failed  units  will 
then  enter  the  repair  pipeline  with  its  associated  shipping 
time  to  the  depot. 

Upon  arrival  at  the  depot  module,  the  units  will  be 
checked  to  verify  that  they  are  failed.  False  removals  are 
identified  at  this  point.  Falsely  removed  units  will  be 
sent  on  the  depot  shipping  point  to  be  shipped  back  out  to 
the  bases  thus  shortening  the  depot  pipeline  time.  The 
repair  process  at  the  depot  will  be  modeled  using  the 
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gathered  inputs.  Once  repaired,  the  units  will  be  subjected 
to  a  quality  check  to  verify  they  have  been  repaired.  Those 
failing  must  go  through  the  repair  process  again.  Those 
passing  will  be  shipped  back  to  the  bases  and  placed  into 
supply.  This  is  the  conceptual  framework  for  the 
development  of  the  computer  model. 

Model  Development 

Appendix  A  contains  the  SLAM  II  network  for  the  basic 
model.  Appendix  B  contains  the  SLAM  II  code  for  the  basic 
model.  The  network  is  broken  into  individual  modules  and 
should  be  referred  to  during  the  description  of  the  modules. 

The  ALE-47  Specification  and  Use  Study  outline  the 
operational  requirements  needed  to  develop  a  conceptual 
model.  The  program  office  determined  these  requirements  to 
be  the  baseline  for  any  studies  conducted,  allowing 
comparison  with  any  other  studies  conducted.  The  model 
needs  to  be  a  macro  view  of  flying  operations  which  generate 
failures  in  the  ALE-47.  At  this  time,  the  F-16  C/D  model  is 
the  only  aircraft  in  the  Air  Force  the  ALE-47  will  be 
operational  on.  The  F-16  C/D  flying  operations  need  to  be 
modeled  in  accordance  with  the  operational  requirements 
given  to  the  program  office.  The  exact  number  of  F-16  C/D 
models  and  their  location  could  not  be  determined  because  of 
classification  so  a  decision  was  made  to  use  the  data 


provided  in  the  use  study  pertaining  to  number  of  systems 


and  locations.  These  values  were  used  in  constructing  the 
model.  In  accordance  with  the  operational  requirements, 
each  aircraft  flies  the  same  number  of  sorties  of  the  same 
length  each  day.  There  should  be  no  differences  between 
bases  since  each  has  the  same  number  of  aircraft  as  defined 
by  the  use  study.  The  only  difference  between  bases  is  the 
shipping  time  to  the  depot.  Mr.  T.  P.  Bramlett,  the  AN/ALE- 
40  Item  manager  at  Warner-Robins  Air  Logistics  Center  was 
contacted  to  provide  shipping  times  for  the  various  bases 
for  the  current  dispenser  system.  He  stated  that  there  were 
essentially  only  shipping  times  for  each  theater  of 
operation.  There  are  three  basic  theaters  of  operation  for 
the  F-16,  the  continental  United  States  (CONUS) ,  the 
European  theater  (USAFE) ,  and  the  Pacific  theater  (PACAF) . 

A  distribution  for  shipping  time  to  and  from  each  theater 
was  not  known  and  data  was  not  readily  available.  Mr. 
Bramlett  stated  that  the  average  shipping  time  for  each 
theater  was  known  along  with  observed  minimums  and  maximums. 
Since  this  data  was  known,  a  triangular  distribucion  could 
be  used  to  quantify  the  shipping  times.  A  triangular 
distribution’s  input  parameters  are  the  minimum  shipping 
time,  the  expected  shipping  time,  and  the  maximum  shipping 
time.  These  turn  out  to  be  very  similar  across  theaters  due 
to  the  availability  of  shipping  by  air.  The  shipping  times 
are  provided  in  the  Table  II. 
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Table  II. 

Theater 

Ship  Times 

(days ) 

Theater 

Mini mum 

Expected 

Max i mum 

CONUS 

8 

10 

30 

USAFE 

8 

12 

30 

PACAF 

8 

16 

30 

(V) 

Since  all  bases  are  modeled  exactly  alike  and  the  only 
difference  being  the  theater  ship  time,  it  was  decided  to 
model  the  system  in  terms  of  theaters  to  simplify  the  model. 

From  a  ma cro  point  of  view,  each  theater  generates  a 
set  number  of  sorties  per  day  based  on  the  sortie  generation 
rates,  the  number  of  aircraft,  and  the  flying  hours  per  day. 
Since  failures  are  dependent  on  the  total  number  of  sorties, 
not  whether  they  were  flown  in  singles,  pairs,  or  four-ship 
formations,  it  was  decided  to  generate  the  daily  sorties  one 
at  a  time.  Each  entity  created  is  a  sortie  requirement 
which  must  be  fulfilled.  The  total  number  of  sorties  which 
must  be  flown  in  any  theater  on  a  flying  day  is  determined 
by  multiplying  the  number  of  aircraft  in  the  theater  by  the 
sortie  generation  rate.  The  number  of  sorties  which  must  be 
launched  each  hour  is  determined  by  dividing  the  total  daily 
sorties  by  the  flying  day  length.  The  time  between  creation 
(TBC)  of  each  entity  is  determined  by  taking  the  reciprocal 
of  the  hourly  launches.  This  generates  the  time  between 
sorties  on  an  hourly  basis  which  is  necessary  for  creation 
of  sorties  by  the  simulation  model.  Sorties  are  created  at 
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a  rate  which  will  satisfy  the  total  daily  sorties  which  must 
be  flown.  Each  theater’s  total  sortie  rate  is  a  global 
variable  whose  value  can  be  easily  switched  at  any  time 
point.  Different  rates  were  developed  for  the  wartime 
model.  These  change  the  total  sortie  rate  during  a  war 
surge . 

Total  sorties  flown  is  based  on  a  limited  flying  day. 
Flying  days  during  peacetime  are  set  at  12  hours  in 
accordance  with  the  operational  requirements  in  the 
specification  (1:150-151).  Operations  close  down  for  the 
night  after  12  hours.  The  model  handles  this  with  a  gate. 
The  day  gate  in  the  model  is  opened  for  12  hours  and  then 
closed  for  12  hours.  After  a  entity  is  created,  the  status 
of  the  day  gate  is  checked.  If  day  is  open  the  entity 
continues.  If  not,  the  entity  is  sent  to  a  terminate  node. 

Peacetime  flying  days  are  limited  to  260  days  per  year 
in  accordance  with  the  ALE-47  Use  Study  (2:8) .  This  works 
out  to  a  5  day  flying  week,  52  weeks  per  year.  The  flying 
week  is  handled  the  same  as  the  flying  day  by  the  model. 

The  week  gate  is  opened  for  120  hours  and  then  closed  for 
48.  The  status  is  checked  by  each  entity  before  it  is 
allowed  to  proceed.  If  open,  the  entity  proceeds.  If  not, 
the  entity  is  routed  to  a  terminate  node. 

If  the  entity  has  passed  this  far  through  the  model 
unobstructed,  it  is  now  a  sortie  that  must  be  flown  to  meet 
requirements.  It  is  assigned  a  set  of  attributes  which 
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determine  whether  it  will  fail  on  the  sortie,  what  its 
shipping  time  to  the  depot  will  be,  and  whether  it  will  be  a 
false  removal.  Since  failures  and  false  removals  are 
percentages,  a  uniform  random  number  between  zero  and  one  is 
assigned  to  these  attributes.  Shipping  time  to  the  depot, 

1 rom  Lae  uepuL,  a, id  the  depot  repair  time  are  all  assumed  to 
be  triangular  distributions.  These  are  assigned  to 
attributes  at  this  point  also.  Assigning  these  values  at 
this  point  allows  repeatability  of  the  runs  using  the  same 
random  number  seeds . 

Once  the  entity’s  attributes  have  been  assigned,  it 
attempts  to  get  an  aircraft  and  fly  a  sortie.  An  assumption 
of  the  model  is  that  any  available  aircraft  can  be  used  to 
fly  a  sortie  when  demanded.  This  approximates  reality. 

This  allows  the  sortie  rate  to  be  met  while  some  aircraft 
are  down  for  various  reasons  including  awaiting  spares. 

This  also  allows  the  base  spares  level  to  be  zero  and  still 
no  sorties  lost  providing  there  is  time  during  the  flying 
day  to  complete  all  missions.  Aircraft  are  modeled  as 
resources  and  are  assigned  to  sorties  through  an  await  node. 
This  allows  any  available  aircraft  to  fulfill  a  sortie 
requirement.  The  status  of  the  resource  is  checked  before 
the  entity  enters  the  await  node.  If  there  are  no  aircraft 
available,  the  entity  is  routed  through  a  lost  sortie 
counter  and  then  terminated.  The  number  of  lost  sorties  is 
collected  and  provided  as  output.  If  an  aircraft  is 
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available,  it  is  utilized  and  departs  on  a  1.5  hour  sortie. 

After  completion  of  the  sortie,  the  system  is  checked 
for  failure.  The  system  breakrate  is  compared  to  the 
uniform  random  number  assigned  to  the  entity  just  after 
creation.  If  the  random  numb.er  is  less  than  or  equal  to  the 
breakrate,  a  failure  has  occurred  and  the  entity  is  routed 
to  the  base  supply  module.  If  greater,  no  failure  has 
occurred  and  the  entity  is  routed  to  a  free  node  which  frees 
the  aircraft  resource  to  fly  another  mission.  The  entity  is 
then  terminated. 

The  base  supply  module  performs  two  functions.  It 
repairs  the  break  on  the  aircraft  with  a  spare  if  available 
and  ships  the  failed  system  to  the  depot.  The  entity  is 
split  down  two  paths  upon  arrival.  One  entity  is  sent  to 
the  shipping  await  node  and  the  other  to  the  spares  await 
node.  If  a  spare  is  available  the  entity  proceeds  through 
the  node  and  frees  the  aircraft  resource  so  another  sortie 
can  be  flown.  If  no  spare  is  available,  the  entity  waits  in 
the  await  node  until  one  arrives  from  the  depot.  Shipping 
occurs  once  a  week  on  a  regular  basis.  Systems  which  fail 
during  the  week  await  shipping.  Shipping  is  handled  by  a 
gate.  The  gate  is  opened  once  every  160  hours  and  frees  ail 
the  entities  to  be  shipped  to  the  depot.  The  frequency  of 
opening  the  gate  is  assigned  to  a  global  variable.  This 
allows  the  shipping  frequency  to  be  varied  for  sensitivity 
analysis.  Entities  are  assigned  a  shipping  time  when 
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created  based  on  the  triangular  distribution  for  the  theater 
in  which  they  reside. 

Upon  arrival  at  the  depot,  a  check  is  performed  to  see 
if  a  false  removal  has  occurred.  The  false  removal  rate  is 
checked  against  a  uniform  random  number  assigned  earlier. 

If  less  than  or  equal  to  the  false  removal  rate,  the  entity 
has  been  falsely  removed  due  to  testability  problems,  and  is 
sent  directly  to  depot  shipping  to  be  shipped  back  to  the 
theater.  If  greater  than  the  false  removal  rate,  the  entity 
proceeds  through  the  depot  repair  process.  The  repair 
process  is  unconstrained,  an  assumption  that  is  also  made  by 
Dyna-METRIC  (16:5) .  Discussion  with  Mr.  Bramlett,  the  ALE- 
40  Item  manager,  confirmed  this  assumption.  Repair  of  the 
current  dispenser  is  basically  unconstrained  depending  on 
the  total  depot  workload  (8).  The  dispenser’s  repair  is 
generally  basic  electronic  repair  which  can  be  handled 
fairly  easily.  Data  was  not  readily  available  on  the  repair 
process  but  could  be  modeled  with  a  triangular  distribution 
based  on  data  provided  by  Mr  Bramlett.  Repair  is  done  on 
the  average  in  26  days,  with  a  minimum  of  10  days  and 
maximum  of  40  days  (8) .  The  repair  time  is  assigned  to  an 
attribute  upon  creation  of  the  entity.  Upon  completion  of 
repair  a  quality  control  check  is  made  to  verify  the  system 
is  operational.  A  uniform  random  number  is  assigned  to  an 
attribute  and  it  is  checked  against  a  rejection  rate  of  one 
percent.  If  the  system  fails  the  test,  the  repair  process 


34 


is  repeated.  Upon  completion  the  random  number  is 
reassigned  and  the  check  made  again.  The  unit  may  fail 
again,  just  as  in  reality,  since  repair  processes  at  this 
level  often  induce  additional  failures. 

Once  the  system  passes  the  quality  check,  it  is  sent  to 
the  depot  shipping  node  where  it  awaits  shipping  back  to  its 
base.  The  shipping  time  is  again  modeled  with  a  triangular 
distribution  but  a  different  value  is  drawn  for  the  return 
shipping  time.  Once  the  entity  arrives  at  the  theater,  it 
frees  a  spare  resource  and  then  is  terminated. 

A  daily  check  is  made  of  the  spares  level  at  each 
theater  and  is  recorded  in  a  tabular  format  for  analysis  in 
a  spreadsheet  or  statistics  software  package. 


Verification  and  Validation 

A  model’s  validity  should  be  determined  with  respect  to 
the  purpose  for  which  it  was  constructed.  It  is  valid  if  it 
IS  accurate  enough  to  provide  acceptable  results  for  the 
purpose  of  the  study.  Validating  a  model  is  part  of  a 
simulation  study.  Validation  can  take  place  in  a  subjective 
or  objective  manner  and  tnere  are  many  techniques  available 
to  do  so.  The  following  is  a  list  of  techniques  (23:33-34) : 

A.  Comparison  to  other  models. 

B.  Degenerate  tests. 

C.  Extreme-condition  tests. 

D.  Face  Validity. 
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E.  Fixed  Values. 

F.  Predictive  Validation. 

Law  and  Kelton  outline  a  three-step  approach  to  validate 
simulations,  consisting  of  constructing  the  model  with  high 
face  validity,  empirically  testing  the  assumptions,  and 
determining  how  representative  the  output  is.  High  face 
validity  during  model  construction  is  obtained  by  drawing  on 
information  from  experts,  using  existing  theories  and 
knowledge,  and  comparing  with  other  models.  Flow  diagrams 
of  the  simulation  model  were  provided  to  people  in  the  ALE- 
47  System  Program  Office  familiar  with  the  maintenance 
system  being  simulated.  As  iterations  in  the  model 
developed  and  further  information  was  needed,  the  same 
personnel  were  consulted  as  to  the  validity  of  the  changes. 
Large,  complex  models  exist  which  simulate  maintenance 
systems.  Dyna-METRIC  is  an  example  of  such  a  model.  Its 
documentation  was  reviewed  to  aid  in  proper  development  of 
the  model.  The  ALE-47  System  Program  Office  determined  this 
is  a  valid  model  for  the  ALE-47  two-level  concept.  Since 
the  model  was  accepted  as  valid  for  the  study,  experiments 
could  be  designed  to  answer  the  research  questions. 

Determining  Steady-State  Conditions 

The  initial  starting  conditions  can  bias  the  output 
from  a  simulation.  The  period  of  time  necessary  for  the 
daily  spare  level  in  each  theater  to  settle  down  to  steady 
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state  must  be  determined  to  eliminate  biases  in  the  output 
data  due  to  the  transient  period.  There  are  two  basic 
approaches  to  eliminate  this  bias.  A  warm-up  period  can  be 
used  or  the  typical  starting  conditions  can  be  input  as  the 
initial  conditions  (20:161).  Determining  typical  starting 
conditions  for  this  system  was  difficult  so  a  warm-up 
approach  was  chosen.  There  are  no  easily  implemented 
methods  for  determining  the  length  of  the  warm-up  period 
(20:161).  This  will  be  accomplished  by  plotting  and 
comparing  spares  level  as  time  advances  over  several  pilot 
runs.  Eight  pilot  runs  of  5  years  length  were  performed  and 
plotted.  A  comparison  was  made  across  the  eight  runs  to 
determine  the  transient  phase.  In  all  oases  it  was  apparent 
steady  state  had  been  achieved  by  4000  hours  (about  six 
months  of  simulated  time) .  As  a  safety  measure,  it  was 
decided  to  extend  the  run-in  length  to  one  year  (8760 
hours) .  Any  transient  effects  should  be  overcome  by  this 
point . 

Experimental  Design 

There  are  four  experiments  in  the  study  which  attempt 
to  answer  the  research  questions.  The  first  is  to  develop  a 
probability  distribution  for  the  monthly  mean  spares  level 
for  each  theater.  The  second  is  to  develop  a  probability 
distribution  for  the  monthly  minimum  spares  level  for  each 
theater.  The  third  is  to  determine  the  effect  of  going  to 
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war  with  its  increased  sortie  rates  on  the  spares  level  of 
each  theater.  The  fourth  la  to  determine  the  weights  of 
each  factor  in  determining  the  response. 

Experiments  one  and  two  can  be  conducted  from  the  same 
set  of  runs.  Thirty  independent  samples  of  spares  level  for 
each  theater  over  a  thirty  day  period  will  be  collected  as 
output  data.  For  experiment  one,  each  sample’s  theater 
daily  values  will  be  averaged  to  provide  a  mean  for  each 
theater  for  each  sample.  The  means  will  be  used  to  develop 
a  probability  distribution  for  the  mean  spares  level  for 
each  theater  for  a  thirty  day  period.  95  percent  confidence 
limits  will  be  placed  on  the  distribution.  This  will  allow 
the  theater  mean  spares  level  in  a  thirty  day  period  to  be 
stated  within  the  confidence  limits. 

For  experiment  two,  the  same  data  can  be  utilized  with 
the  minimum  value  from  each  theater  in  each  sample  being 
identified.  Theory  states  that  extremes  are  known  to  follow 
one  of  three  non-normal  limiting  distributions  (9:199). 
Theory  also  states  that  minimums  (the  smallest  extremes)  are 
known  to  follow  the  Weibull  distribution  (9:200) .  The 
thirty  minimum  values  from  each  of  the  theaters  will  be  used 
to  build  a  distribution  which  will  be  tested  against  the 
Weibull  distribution.  Ninety-five  percent  confidence  limits 
will  be  placed  on  the  distribution.  If  the  distribution 
holds,  this  allows  the  theater  minimum  spares  level  in  a 
thirty  day  period  to  be  stated  within  the  confidence  limits. 
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If  the  distribution  cannot  be  stated,  nonparametric  methods 
will  be  used  for  analysis  of  the  minimums. 

Experiment  three  will  attempt  to  quantify  the  effects 
of  the  wartime  surge.  The  average  amount  of  time  it  takes 
the  system  to  settle  down  to  a  new  steady  state  will  be 
determined  in  addition  to  the  drop  in  the  average  daily 
spares  level.  The  daily  spares  level  will  be  collected 
throughout  the  length  of  the  run.  Several  runs  will  be 
made.  An  average  level  for  a  thirty  day  period  in  peacetime 
just  before  the  war  surge  will  be  collected.  A  similar 
average  level  for  a  thirty  day  period  after  wartime  steady 
state  has  been  reached  will  be  collected.  These  two 
averages  will  be  used  to  quantify  the  drop  in  average  dally 
spares  level  for  a  thirty  day  period.  The  time  taken  to 
settle  into  a  wartime  steady  state  will  be  averaged  across 
the  runs.  These  two  measures  will  be  used  to  quantify  the 
effects  of  the  wartime  surge  on  the  system. 

A  full  2*^  factorial  design  will  be  used  to  determine 
the  relative  effects  the  three  input  factors  have  on  sortie 
generation  capability  in  experiment  four.  The  2^ 
experimental  design  varies  each  factor  at  two  levels  and 
since  there  are  three  input  factors  this  will  require  eight 
separate  runs  replicated  several  times.  Comparison  of  the 
outputs  will  allow  quantification  of  the  effects  of  each 
ff  ’tor  on  the  spares  level.  In  addition,  the  effects  of 
interaction  between  the  factors  on  spares  level  can  be 
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determined  from  the  full  f actor ial  design  (14:116).  The 
effects  of  each  factor  and  the  interactions  will  be  tested 
for  s i gn i f i cance .  This  data  can  be  used  for  tradeoff 
analyses  between  the  factors  if  one  does  not  achieve  its 
projected  level.  The  data  will  also  prove  useful  if  other 
program  parameters  change  and  force  changes  in  these 
factors . 

Summary 

This  methodology  provides  the  basis  of  a  technique  to 
evaluate  the  operational  suitability  of  the  two-level 
maintenance  concept  for  systems  which  are  still  under 
development.  The  results  of  this  experiment  provides 
operational  suitability  information  to  the  ALE-47  Program 
Office  for  use  in  their  maintenance  concept  decision.  It 
allows  them  to  predict  the  minimum  and  average  spares  levels 
for  a  thirty  day  period.  The  effects  of  the  wartime  sortie 
rates  and  of  each  factor  on  the  spares  level  can  be 
quantified  for  the  system.  This  methodology  does  not 
provide  a  generic  model  but  does  provide  a  framework  which 


can  be  customized  for  any  system  under  development. 


I n  t  r :  -uction 

This  chapter  describes  the  results  obtained  by  using 
the  model  and  conducting  the  experiments  outlined  in  the 
previous  chapter.  The  methods  used  to  analyse  the  results 
are  discussed  with  the  results.  In  each  experiment,  it  was 
necessary  to  achieve  steady-state  conditions  before 
collecting  data.  It  was  also  necessary  to  obtain 
independent  observations.  The  methods  described  in  the 
previous  chapter  were  used  to  fulfill  these  requirements. 
Each  experiment  is  discussed  in  the  order  performed. 

It  should  be  noted  that  no  sorties  were  lost  due  to 
inadequate  spares  in  any  of  the  simulation  runs.  The 
theaters  were  all  capable  of  the  sortie  generation  rate.  A 
run  with  a  failure  rate  high  enough  to  cause  a  failure  every 
flight  was  made  to  verify  that  the  lost  sortie  counter  was 
operating  properly.  The  data  confirmed  the  counter  was 
operating  properly.  The  assumption  that  any  aircraft 
available  can  fill  the  sortie  requirement  appears  to  be  the 
reason  no  sorties  were  lost. 

Analysis  of  Means 

The  average  monthly  spares  available  in  each  theater 
after  the  system  is  in  steady  state  is  of  interest  since  the 


ability  of  a  unit  to  wage  war  depends  on  its  available 


supplies.  Characterization  of  the  average  monthly  spares 


level  provides  one  measure  of  the  operational  suitability  of 
the  system  with  the  two-level  maintenance  concept.  A  95 
percent  confidence  Interval  will  be  developed  around  the 
average  monthly  spares  level  for  each  theater  under 
peacetime  operating  conditions. 

Thirty  replications  of  the  basic  model  were  made.  Each 
replication  resulted  in  data  which  consisted  of  one  month 
(30  days)  of  daily  observations  of  each  theater’s  spares 
level.  Each  replication’s  data  points  were  averaged  to 
produce  a  mean  for  each  theater  for  that  particular 
replication.  The  result  is  thirty  means  for  each  theater. 
Each  theater’s  means  were  analyzed  separately.  The  means 
are  listed  in  Appendix  C.  The  central  limit  theorem  states 
that 


if  random  samples  of  n  measurements  are  repeatedly 
drawn  from  a  population  with  a  finite  mean  pi  and  a 
standard  deviation  <t  ,  then,  when  n  is  large,  the 
relative  frequency  histogram  for  the  sample  means 
(calculated  from  the  repeated  samples)  will  be 
approximately  normal  ( be  1 1  -  s  haped  )  with  mean  ^l  and 
standard  deviation  a//n  (19: 109) . 


Assuming  independent  data  was  generated  as  discussed 
earlier  and  using  the  central  limit  theorem,  the  data  should 
follow  the  normal  distribution.  To  verify  that  30 
replications  were  enough  to  satisfy  confidence  requirements, 
Equation  (1)  was  used. 

R/r=(d/D)^  (1) 
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where  R  equals  the  number  of  replications  needed  for  the 
desired  confidence,  r  equals  the  number  of  pilot 
replications  performed,  d  equals  the  confidence  interval 
width  calculated  from  the  pilot  runs,  and  D  equals  the 
desired  confidence  interval  width  (22) .  The  pilot  run 
confidence  interval  width,  d,  is  calculated  using  a  t 
statistic  confidence  interval.  The  estimate  of  the  mean 
standard  deviation,  produced  by  the  pilot  run,  is  multiplied 
by  the  t  value  for  the  confidence  desired  and  the  number  of 
replications.  This  value  is  subtracted  from  the  mean  to 
produce  the  lower  confidence  limit  and  added  to  the  mean  to 
produce  the  upper  confidence  limit.  The  difference  between 
the  limits  is  the  confidence  interval  width.  The  desired 
confidence  Interval  width  was  chosen  to  be  4  or  plus  or 
minus  two  spares.  The  number  of  replications  necessary  to 
obtain  a  95  percent  confidence  Interval  on  the  means  of  each 
theater  are  given  in  Table  III. 

Table  III.  Number  of  Replications  Needed 

Theater  Number  of  Replications 

CONUS  29.7 

USAFE  27 . 1 

PACAF  10.4 

Halving  the  desired  confidence  interval  width  to  2  requires 
over  100  replications  for  the  CONUS  and  USAFE  theaters.  A 
confidence  interval  width  of  4  is  reasonable  for  this  study. 
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so  it  was  decided  not  to  expend  additional  computer  time  and 
resources  for  a  smaller  confidence  interval  width. 

To  graphically  portray  the  shape  of  the  data,  a 
frequency  histogram  was  generated  for  each  theater’s  mean 
monthly  spares  level.  The  histograms  are  displayed  in 
F i gur es  1 ,  2 ,  and  3 . 

Just  reviewing  the  frequency  histograms  for  a  bell 
shape  does  not  constitute  a  test  for  the  normal 
distribution.  Additionally,  the  data  was  checked  for 
normality  using  the  Wi Ik-Shapiro/Ranki t  Plot  (24:8.4) .  In 
the  Wi Ik-Shap i r o /Rank i t  Plot,  the  sample  values  are 
reordered  according  to  their  rank  to  generate  order 
statistics.  The  expected  value  of  each  order  statistic  is 
defined  as  its  'rankit'.  The  rankits  follow  the  normal 
distribution  with  mean  0  and  variance  1.  The  rankits  are 
then  plotted  against  the  order  statistics.  If  the  sample 
follows  the  normal  distribution,  the  plot  should  show  a 
linear  trend  with  only  slight  random  error  (24:0.5) .  The 
Wi  Ik-Shapiro/Ranki t  Plots  for  each  theater  are  shown  in 
Appendix  D.  Each  theater’s  plot  indicates  the  normal 
distribution  is  an  acceptable  distribution  for  the  mean 
monthly  spares  level  as  was  expected  by  the  central  limit 
theorem . 

The  means  and  standard  deviations  for  each  theater  are 
listed  in  Table  IV.  The  mean  for  each  theater  is  the 
average  of  the  mean  from  each  replication.  The  standard 
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Figure  1.  CONUS  Average  Monthly  Spares  Histogram 


Figure  2.  USAFE  Average  Monthly  Spares  Histogram 
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Figure  3.  PACAF  Average  Monthly  Spares  Histogram 


deviation  of  the  mean  is  the  sample  standard  deviation 
divided  by  the  square  root  of  30,  the  number  of  samples. 


Table  IV.  Theater  Statistics  (Basic  Model) 


Theater 

Mean 

Standard  Deviation 

CONUS 

4  1.37 

0 . 9742 

USAFE 

42 . 56 

0 . 9289 

PACAF 

20 . 79 

0 . 5753 

An  additional  run  of  10  years  in  length  was  made.  A  check 
of  the  normality  of  the  system  can  be  made  by  plotting  the 
number  of  days  a  particular  spares  level  was  observed  over 
the  length  of  the  run.  The  first  year’s  data  was  dropped 
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from  the  analysis  as  before  due  to  system  warm-up  time.  The 
frequency  histograms  for  each  theater  are  shown  in  Appendix 
E. 

Since  it  can  be  assumed  that  the  distribution  of  the 
mean  of  the  spares  levels  in  each  of  the  theaters  follows 
the  normal  distribution,  a  confidence  interval  can  be 
constructed  around  the  mean.  This  will  be  used  to  quantify 
the  true  monthly  mean  spares  level  within  limits.  A  95 
percent  confidence  interval  was  chosen  for  the  experiment. 
The  upper  and  lower  confidence  limits  are  given  by  Equation 
2  . 

(y  -  1.96ct  ,  y  +  1 . 96<J  )  (2) 

where  y  is  the  observed  sample  mean  and  a  is  the  sample 
standard  deviation.  Table  V  summarizes  the  confidence 
limits  at  the  95  percent  confidence  level  for  each  of  the 
three  theaters . 


Table  V.  Confidence  Interval  Limits  (Means) 


Theater 

Lower 

CONUS 

39 . 46 

USAFE 

40 . 74 

PACAF 

19.66 

limit  Upper  limit 

43 . 23 
44 . 38 
21.92 


The  analysis  was  performed  under  peacetime  conditions 
and  indicates  the  average  monthly  spares  level  should 
adequately  support  operations  in  each  theater. 
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The  confidence  interval  about  the  mean  for  each  theater 


is  important  in  determining  the  operational  suitability  of 
the  two-level  maintenance  concept  since  the  number  of  spares 
available  significantly  impacts  the  warfighting  capability 
of  an  operational  unit.  The  average  number  of  spares 
expected  to  be  in  any  theater  in  any  given  month  can  be  an 
important  planning  tool. 

Sensitivity  of  Means  to  Shipping  Frequency.  Since  the 
frequency  of  shipping  from  each  theater  and  to  each  theater 
could  not  be  accurately  quantified,  a  sensitivity  analysis 
was  conducted  by  varying  the  frequency  of  shipping  in  the 
model.  Two  additional  runs  were  made  of  thirty  replications 
each  using  the  same  data  collection  scheme.  In  the  first 
run,  shipping  occurs  every  two  days  from  each  theater  and 
from  the  depot.  In  the  second,  shipping  occurs  every 
fourteen  days.  The  means  and  standard  deviations  from  the 
runs  for  each  theater  are  listed  in  Table  VI. 

Table  VI.  Theater  Statistics  for  Various 
Shipping  Times 


Ship  Time 

Theater 

Mean 

Standard  Deviation 

2  days 

CONUS 

44.81 

0 . 6452 

USAFE 

43 . 68 

0 . 8873 

PACAF 

21.41 

0 . 6648 

14  days 

CONUS 

38 . 65 

0 . 9848 

USAFE 

37 . 73 

0 . 8451 

PACAF 

18.58 

0.4172 
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To  determine  if  there  is  a  significant  difference  from  the 
seven  day  shipping  frequency  assumption  used  in  the  basic 
model,  a  paired  sample  t-test  was  used  twice.  The 
comparison  was  made  between  the  two  and  seven  day  models  and 
the  seven  and  fourteen  day  models.  The  null  hypothesis  in 
both  tests  was  that  the  difference  between  the  two  means  was 
equal  to  zero.  The  alternate  hypothesis  is  that  the 
difference  between  the  means  is  not  equal  to  zero,  thus 
implying  that  there  is  some  significant  difference  between 
the  means.  The  confidence  level  for  this  test  was  95 
percent.  The  test  statistic  is  defined  by  Equation  (3) . 

t  =  d  /  (s/Zn)  (3) 

where  d  is  the  difference  between  the  means,  s  is  the 
standard  deviation  of  the  differences  in  the  means,  and  n  is 
the  number  of  means  (19:196).  The  software  package 
STATISTIX  was  used  to  perform  this  test.  The  results  are 
showed  significant  differences  in  the  differences  of  the 
means  for  all  combinations  except  USAFE  and  PACAF  between 
the  seven  and  two  day  models.  The  results  are  stated  in 
Appendix  F.  For  the  purposes  of  this  study,  the  seven  day 
model  results  will  be  used.  There  are  other  simplifying 
assumptions  employed  in  the  model  so  this  should  be 
acceptable.  Any  additional  studies  should  characterize  the 
shipping  frequency  more  accurately  if  possible. 

The  mean  monthly  spares  level  is  only  one  measure  of 
operational  suitability,  however.  Another  measure  of 
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operational  suitability  which  may  be  equally  important  is 
the  minimum  monthly  spares  level  in  each  theater. 

Analysis  of  Mini mums 

The  average  spares  level  for  any  theater  appears  to  be 
a  good  measure  of  suitability  but  since  the  Air  Force  would 
like  to  be  always  ready  to  go  to  war  the  monthly  minimum 
spares  level  would  be  of  more  interest  in  order  to  plan  for 
all  contingencies.  Theory  states  that  minimums  follow  the 
Weibull  distribution,  but  the  Weibull  can  assume  the  shape 
of  many  other  distributions  including  the  normal  and 
exponential  (13).  The  same  data  used  in  the  analysis  of 
means  is  used  in  the  analysis  of  the  minimums.  The  minimum 
value  from  each  theater  in  each  replication  was  identified. 
The  thirty  minimums  for  each  theater  then  constituted  the 
data  set  for  analysis.  First,  a  frequency  histogram  for 
each  theater  was  constructed  to  graphically  portray  the 
distribution  shape.  The  three  histograms  are  shown  in 
Figures  4,  5,  and  6.  The  underlying  distribution  of  the 
minimums  is  not  apparent  from  the  histograms.  The  minimums 
from  each  theater  were  tested  for  conforming  to  the  Weibull 
distribution  using  the  Kolmogorov-Smirnov  Test  in  AID,  the 
SLAM  II  data  analysis  package.  Each  set  of  data  accepted 
the  hypothesis  that  the  data  followed  the  Weibull. 
Additionally,  the  W1 Ik-Shaplro/Rankl t  Plot  was  used  to  check 
for  normality.  The  Rankit  Plots  for  each  theater  are  shown 
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Figure  4.  COK’US  Minimum  Monthly  Spares  Level 
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Figure  5.  USAFE  Minimum  Monthly  Spares  Level 
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figure  6.  PACAF  liiniaum  Monthly  Spares  Level 

in  Appendix  D.  In  all  three  theaters,  the  plots  indicate 
the  normal  distribution  is  also  a  possible  model.  If  the 
minimums  follow  the  normal  distribution,  means  and  standard 
deviations  for  each  theater  can  be  calculated.  Table  VII 
summarizes  the  means  and  their  standard  deviations  of  the 
minimums  for  each  theater. 

Table  VII.  Theater  Statistics  (Minimums) 


Theater 

Mean 

Standard  Deviation 

CONUS 

37.63 

0 . 996 

USAFE 

39 . 20 

0.919 

PACAF 

la  .  47 

0.711 
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These  represent  the  expected  value  of  each  theater's 
monthly  minimums.  If  the  normal  is  assumed,  a  95  percent 
confidence  interval  can  be  developed  around  each  theater’s 
mean  in  order  to  make  statements  about  the  mean  monthly 
minimum  spares  level.  The  confidence  interval  is  given  by 
the  formula  (y  -  1.96c  ,  y  +  1.96c  ).  Table  VIII  summarizes 
the  95  percent  confidence  intervals  for  each  theater. 

Table  VIII.  Confidence  Interval  Limits 


(Means  of  Minimums) 

Theater 

Lower  Limit 

Upper  Limit 

CONUS 

35 . 63 

39 . 58 

USAFE 

37 . 40 

41.00 

PACAF 

17.08 

19 . 86 

It  should  be  noted  that  zero  does  not  fall  within  any  of  the 
intervals  even  at  the  95  percent  confidence  level.  This 
implies  that  the  chances  of  the  spares  level  being  zero  is 
very  small  in  each  of  the  theaters. 

Since  the  minimums  passed  tests  for  different 
underlying  distributions,  it  is  possible  the  data  set  is  too 
small  to  accurately  determine  the  distribution.  Any 
analysis  performed  with  this  data  should  employ 
nonparame tr i c  statistics  since  the  underlying  distribution 
cannot  be  accurately  determined. 
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Analysis  of  Surge 


Peacetime  represents  a  fairly  stable  environment  for 
the  operation  of  the  ALE-47,  but  wartime  changes  the 
aircraft  sortie  rates  significantly.  The  higher  sortie 
rates  cause  system  failure  more  often  and  cause  the  whole 
maintenance  system  to  settle  to  a  new  mean  level  in  each 
theater.  The  objective  of  this  experiment  is  to  quantify 
the  change  in  the  mean  spares  level  from  peacetime  flying 
rates  to  wartime  surge  rates.  The  change  is  examined  in  two 
dimensions.  The  first  is  the  average  drop  in  the  mean 
spares  level  from  peacetime  to  wartime.  The  second 
dimension  is  the  amount  of  time  necessary  for  the  system  to 
settle  down  to  a  steady  mean  spares  level. 

An  additional  module  was  added  to  the  basic  model  which 
started  the  war  surge  after  one  year  (8760  hours) .  It 
changed  the  sortie  rates  for  each  theater  to  the  war  surge 
rates  stated  in  the  specification.  To  accurately  quantify 
the  drop,  the  number  of  spares  available  in  each  theater  was 
set  to  100  percent  in  these  runs  of  the  model  as  opposed  to 
20  percent,  the  proposed  rate.  This  precludes  the  chance 
that  the  surge  drop  would  cause  the  level  to  drop  to  zero  or 
below.  Since  the  model  does  not  collect  values  below  zero, 
it  would  be  impossible  to  quantify  the  drop  if  it  went  below 
zero.  Starting  with  100  percent  spares  allows  the  effects 
of  the  surge  to  be  quantified  by  dropping  from  an  average 
presurge  level  to  an  average  post  surge  level.  The 
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parameter  of  interest  is  the  amount  of  the  drop,  not  the  pre 
and  post  levels  themselves.  The  data  collection  scheme  was 
changed  to  record  daily  values  of  each  theater’s  spares 
level.  The  daily  values  for  the  five  runs  were  averaged  to 
produce  an  average  daily  spares  level  for  each  theater. 

Plots  showing  the  effect  of  the  war  surge  on  each  theater 
follow  in  Figures  7,  3,  and  9. 

The  presurge  average  theater  spares  level  is  calculated 
using  the  daily  spares  levels  observed  from  4000  to  8760 
hours.  The  war  surge  commences  at  the  8760  hour  point.  Each 
theater’s  plot  is  examined  to  determine  an  approximate  point 
in  time  at  which  a  new  steady  state  is  achieved.  In  the 
CONUS  theater,  the  graph  indicates  post  surge  steady  state 
is  achieved  at  approximately  the  11,500  hour  point.  A 
postsurge  average  spares  level  was  calculated  with  the  data 
from  that  point  to  the  20,000  hour  point  which  ends  the 
simulation  run.  This  postsurge  level  is  then  subtracted 
from  the  presurge  level  to  develop  an  average  drop  due  to 
the  war  surge  for  each  theater.  CONUS  restabilisation 
occurs  at  approximately  11,500  hours  which  equates  to 
approximately  113  days.  This  is  the  number  of  days 
necessary  for  the  system  to  stabilize  into  steady-state  at 
the  new  sortie  generation  rate  of  1.5  sorties  per  aircraft 
per  day.  The  stabilization  point  for  USAFE  is  approximately 
11,100  hours.  This  equates  to  approximately  98  days  from 
the  start  of  the  surge.  The  PACAF  stabilization  point 
occurs  at  approximately  11,600  hours.  This  equates  to 
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Fijjurc-  7.  CONUS  War  Surge  Effects  on  Spares  Level 


gum  8.  USAFE  War  Surge  Effects  ou  Spares  Level 


approximately  118  days.  Table  IX  provides  the  pre  and  post 
surge  average  spares  levels  for  each  theater  as  well  as  the 
average  drop  in  spares  level  for  each  theater. 

Table  IX.  Average  Pre  and  Post  Surge  Spares  Levels 

Theater  Presurge  Postsurge  Average  Drop 

CONUS  35'?. 26  314.48  42.78 

USAFE  356.88  310.34  46.54 

PACAF  141.32  122.85  18.47 

Effect  on  Means.  Applying  the  average  drop  to  the 
average  monthly  spares  level  in  peacetime  Indicates 
shortages  may  occur  providing  no  other  parameters  which 
affect  the  spares  level  are  changed.  Since  the  underlying 
distribution  of  the  average  monthly  spares  level  in  each 
theater  was  determined  to  be  normal,  Z  scores  were  used  to 
determine  the  probability  of  the  average  monthly  spares 
level  being  greater  than  the  average  drop  expected  in  the 
war  surge.  Z  values  were  calculated  for  the  drops  in  each 
theater  using  equation  (4) 

Z=(|i-y)/  cr  (4; 

where  Z  is  the  calculated  Z  score  to  be  looked  up  in  a 
table,  is  the  sample  mean  for  each  theater,  y  is  the 
average  drop  for  that  theater,  and  a  is  the  standard 
deviation  of  the  mean  for  that  theater.  Z  tables  provide 
the  probabilities  which  correspond  to  those  values  The 
portion  underneath  the  probability  curve  which  is  greater 
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than  the  Z  value  constitutes  the  probability  of  the  monthly 
spares  level  for  that  theater  being  greater  than  the  average 
drop.  Table  X  summarizes  the  probability  of  the  average 
monthly  spares  level  being  greater  than  the  average  drop 
calculated  for  the  theater. 


Table  X.  Probabilities  of  Average  Spares  Levels 
Being  Greater  than  Average  Drop 

i 

Probability  of  Mean 
Theater  Spares  Level  >  Drop 


CONUS 

USAFE 

PACAF 


.  395 
.2  17 
.  769 


Effect  Oil  Minimums.  Since  the  underlying  distribution 
of  the  minimums  could  not  be  accurately  determined,  a  non 
parametric  approach  was  used  to  evaluate  the  effects  of  the 
surge  given  the  monthly  spares  level  is  near  its  minimum 
level.  The  purpose  of  this  analysis  is  to  predict  the 
percentage  of  the  values  which  are  less  than  a  certain 
value.  This  can  be  done  using  the  binomial  distribution. 
Binomial  data  is  characterized  by  either  being  in  a 
specified  interval  or  not.  The  chance  it  will  fall  in  the 
specified  Interval  is  de.ined  as  p.  An  estimate  of  p  must 
be  obtained.  The  number  of  values  in  each  theater  which 
were  less  than  the  average  drop  for  that  theater  were 
counted.  This  number  was  divided  by  the  number  of 


observations  in  each  theater.  The  number  of  observations  in 


each  theater  is  thirty  in  this  study.  This  division 


provides  an  estimated  p  value  which  can  be  used  to  develop  a 
probability.  Welch  states  that  if  Mp(l-p)  >  9,  the  normal 
approximation  to  the  binomial  may  be  used  (25:287) .  He  also 
states,  "this  rule  may  be  too  conservative  for  some 
s i mul at i on  work  (25:287).’  Ott  states  that  if  np  <  5  or 
n(l-p)  <  5,  the  normal  approximation  to  the  binomial  may  be 
unsat i s  f  ac tory  (19:117).  F  or mul as  for  the  calculation  of 
upper  and  lower  p  values  within  confidence  limits  are  also 
given  in  Welch  (25:287) .  The  p  values  and  their  lower  and 
upper  limits  are  given  in  Table  XI. 


Table  XI.  P  Values  and  Confidence  Limits 
for  Average  Minimum  Spares  Levels 
Being  Less  than  Average  Drop 


Theater 

Lower 

p  Value 

Upper 

CONUS 

0.613 

0 . 800 

0.812 

USAFE 

0.313 

0 . 967 

0 . 980 

PACAF 

0 . 292 

0 . 467 

0 . 479 

These  values  represent  the  probability  of  not  having 
enough  spares  to  handle  the  average  war  surge  drop  if  the 
spares  level  is  at  the  average  minimum  value.  This 
indicates  that  the  ability  of  the  theaters  to  surge  in  any 
given  month  can  be  severely  limited  by  the  system  when  at 
minimum  values. 

In  order  to  characterize  the  drop  due  to  the  surge,  it 
was  decided  to  continue  to  fly  the  same  number  of  aircraft 


in  each  theater  at  the  higher  rates.  In  reality,  planes 
from  the  CONUS  would  most  likely  deploy  to  the  theater  of 
conflict.  The  surge  may  be  looked  at  as  just  a  training 
exercise  also.  This  point  of  view  would  allow  the  aircraft 
to  remain  in  their  normal  operating  theater.  Some  other 
items  which  would  definitely  affect  the  spares  level  in  a 
real  conflict  would  be  aircraft  attrition,  production  of 
additional  spares  for  war,  and  speeding  up  of  the  shipping 
and  depot  repair  times. 

Analysis  of  Factorial  Experiment 

The  full  factorial  experiment  is  used  to  examine  the 
contribution  of  each  fac  or  and  the  interactions  between 
them  in  determining  the  average  monthly  spares  level  in  each 
theater.  To  generate  the  data  for  analysis,  eight  separate 
runs  are  made  with  the  factors  varied  on  a  high  and  low 
basis.  Thirty  replications  of  each  run  were  made.  Each  run 
has  specific  high  and  low  value  for  reliability, 
testability,  and  spares  purchase.  Table  XII  lists  the 
values  for  each  factor  in  each  of  the  runs.  The  high  and 
low  values  for  each  are  marked  with  +  and  -  signs 
respect i ve ly . 


62 


Table  XII.  Factor  Values 


Number 

Reliability 

Testabi 1  i 

ty 

Spares  Level 

1 

. 000803 

(  +  ) 

.01 

(  +  ) 

10*/. 

(-) 

2 

. 000876 

{  +  ) 

.  20 

(-) 

10% 

(-) 

3 

. 000803 

(  +  ) 

.01 

(  +  ) 

30% 

(  +  ) 

4 

. 000876 

(  +  ) 

.  20 

(-) 

30% 

(  +  ) 

5 

. 002949 

(-) 

.01 

(  +  ) 

10% 

(-) 

6 

. 003504 

(-) 

.  20 

(-) 

10% 

(-  ) 

7 

. 002949 

(-) 

.01 

(  +  ) 

30% 

(  +  ) 

a 

. 003504 

(-) 

.  20 

(-) 

30% 

(+) 

Low  numerical  values  of  reliability  and  testability  are 
actuall;,  higher  levels  of  the  factors  because  of  the  way 
they  are  incorporated  in  the  model.  Higher  reliability  is 
modeled  with  a  lower  breakrate  and  higher  testability  with  a 
lower  false  removal  rate. 

Using  these  values,  simulation  runs  were  made  and 
average  monthly  values  for  each  theater  were  calculated 
using  the  same  procedure  as  in  the  analysis  of  means.  These 
average  values  were  then  used  to  calculate  the  contribution 
of  each  factor  toward  the  average  monthly  spares  level. 
Equations  5-11  describe  how  the  contribution  of  each  factor 
is  calculated. 


63 


R  =(l  +  2  +  3  +  4-  5-  6-  7-8)/4  (5) 

T  =(l  +  3  +  5  +  7-  2-  4-  6-8)/4  (6) 

SL  =(3  +  4  +  7  +  8-  l-  2-  5-6)/4  (7) 

R/SL  =(3  +  4-  a-  7-  2-  l  +  5  +  6)/4  (8) 

R/T  =(6-2  +  8-  4-  5+  l-  7  +  3)/4  (9) 

T/SL  =  (6  +  2-  8-  4-  5-1  +  7  +  3)  /  4  (10) 

R/T/SL  =  (2+8-6-4+5-1-7+3)  /4  (11) 

where  R  is  reliability,  T  is  testability,  and  SL  is  the 
spares  purchase  size.  The  terms  with  slashes  indicate  the 
interactions  between  two  or  more  of  the  factors.  The 
numbers  within  the  parentheses  represent  the  simulation  run 
number.  The  summation  and  subtraction  of  the  runs 
represents  the  differences  in  the  planes  formed  in  space. 
The  result  is  divided  by  four  because  the  result  represents 
the  summation  of  differences  at  four  points  in  space.  The 
contributions  of  each  of  the  factors  and  ^he  interactions 
across  each  theater  are  summarized  In  Table  XIII. 
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Table  XIII.  Factor  and  Interactions  Contributions 


Factor 

CONUS 

USAFE 

PACAF 

R 

35.15 

36 . 17 

16.39 

T 

1 . 89 

2 . 59 

-0 . 48 

SL 

58 . 72 

58 . 25 

30 . 36 

R/SL 

15.72 

16 . 06 

5 . 45 

R/T 

-2.70 

-1.79 

-0.61 

T/SL 

2.01 

1  .  84 

-0 . 03 

R/T/SL 

-2 . 57 

-2 . 54 

-0.16 

These  contributions  can  be  utilized  as  fi  coefficients  to 
develop  a  regression  equation  that  predicts  the  average 
monthly  spares  level.  In  this  case,  the  program  office 
would  like  to  know  approximately  the  relative  contribution 
of  each  factor  for  planning  purposes.  The  amount  of  spares 
purchased  appears  to  have  the  greatest  positive  effect  on 
the  average  monthly  spares  available  in  each  theater. 
Reliability  is  the  second  most  important,  followed  by  the 
interaction  between  reliability  and  spares  purchased.  To 
test  the  significance  of  each  of  the  effects,  a  multiple 
regression  model  which  included  all  of  the  factors  and  their 
interactions  was  proposed.  The  software  package  STATISTIX 
was  used  to  run  a  multiple  regression  on  this  model.  The 
input  values  for  each  of  the  factors  and  the  interactions 
was  either  a  negative  one  (-1)  or  a  positive  one  (1) .  These 
values  represent  the  high  and  low  values  used  on  each  of  the 
factorial  simulation  runs.  A  matrix  was  developed  for  each 
theater  which  included  the  response  from  each  run  and  the 
associated  factor  and  interaction  values  for  that  run 
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(17:315-317).  STATISTIX  provides  a  t  test  for  each  of  the 
factors  and  interactions  significance.  The  results  for  each 
theater  are  provided  in  Appendix  Q.  The  results  showed 
slight  differences  in  the  PACAF  model  verses  the  other  two. 
In  PACAF,  four  factors  were  determined  to  be  insignificant. 
They  were  Testability,  the  Re 1 i ab i 1 i ty /Tes tab i 1 i ty 
interaction,  the  Testability/Spares  Purchase  interaction, 
and  the  Re  1 i ab i 1 i ty /Tes tabi 1 i ty / Spar es  Purchase  Interaction. 
Only  Testability  was  borderline  insignificant  in  the  other 
two  theaters . 

Summary 

This  chapter  presented  the  four  experiments  which  were 
used  in  the  research.  The  procedures  to  produce  the  data 
and  analyze  the  data  were  discussed.  Assumptions  were 
verified  and  results  stated  for  each  experiment.  The 
results  will  be  used  to  draw  conclusions  and  make 
recommendations  to  the  program  office.  The  results  will 
also  be  used  as  a  basis  for  re<^ommendat i ons  for  further 
study.  Chapter  Five  will  discuss  how  the  results  support 
the  research  objectives  and  questions. 
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V.  Conclusions  and  Recommendations 


Summary  of  Research 

The  objective  of  this  research  was  to  develop  and 
demonstrate  a  technique  to  evaluate  the  operational 
suitability  of  proposed  maintenance  concepts  for  new 
avionics  systems.  The  system  chosen  for  demonstration 
purposes  was  the  ALE-47  Countermeasures  Dispenser  System. 

The  four  research  questions  focused  on  developing 
probability  distributions  for  average  and  minimum  spares 
levels,  quantifying  the  effect  of  the  war  surge  on  those 
levels,  and  quantifying  the  effects  of  the  design  factors  on 
the  operational  suitability  of  the  system. 

A  simulation  model  was  developed  in  SLAM  II  which  used 
as  input  the  operational  requirements  of  the  system  as 
stated  in  the  specification.  Approximate  numbers  of 
aircraft  and  spares  were  used  based  on  program  information. 
The  reliability  levels  stated  in  the  warranty  were  used  for 
field  reliability. 

Probability  distributions  were  developed  for  the 
average  monthly  spares  level  and  minimum  monthly  spares 
level  in  each  theater.  The  effect  of  the  war  surge  in  each 
theater  was  determined  in  terms  of  the  average  drop  in 
spares  level  and  the  length  of  time  necessary  for  the  system 
to  settle  down  into  steady  state  conditions  after  the  surge. 


A  full  factorial  experiment  was  used  to  quantify  the 
contribution  of  each  factor  to  the  average  monthly  spares 
level  in  each  theater. 

Conclusions 

The  average  monthly  spares  level  for  each  theater  can 
be  described  with  a  normal  distribution.  A  mean  and 
standard  deviation  was  developed  for  the  distribution  with 
confidence  limits  placed  around  the  mean.  The  average 
number  of  spares  for  a  month  can  be  quantified  with  this 
value  for  each  theater.  These  values  give  an  indication  of 
the  operational  suitability  of  the  system  with  the  two-level 
maintenance  concept.  These  by  themselves  are  not  a 
complete  view  of  the  operational  suitability  since  the 
average  is  just  that,  an  average.  Maintenance  personnel  are 
most  interested  in  the  chance  of  running  short  of  spares. 
This  is  tied  more  closely  to  the  minimum  value  observed  in  a 
period  of  time. 

Theory  states  that  minimums  follow  the  Weibull 
distribution.  In  this  study  the  minimums  passed  tests  for 
both  the  normal  and  weibull  distributions.  The  Weibull 
distribution  can  take  the  form  of  a  normal  distribution  so 
the  theory  may  still  be  accurate.  The  minimum  values  were 
significantly  lower  than  the  average  values  and  should  be 
used  for  a  worst  case  analysis  of  warfighting  capability. 

The  average  and  the  minimum  values  only  give  an  indication 
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that  normal  peacetime  operations  with  the  two-level 
maintenance  concept  do  not  unduly  tax  the  available  spares. 
Since  these  were  developed  under  peacetime  conditions,  an 
important  measure  is  what  effect  these  peacetime  levels  have 
on  the  warfighting  capability. 

The  war  surge  caused  a  significant  average  drop  in 
spares  level  from  peacetime  to  wartime  in  each  theater. 
Applying  the  average  drop  to  the  average  monthly  spares 
level  in  peacetime  indicates  shortages  may  occur  providing 
no  other  parameters  which  affect  the  spares  level  are 
changed.  Since  the  underlying  distribution  of  the  average 
monthly  spares  level  in  each  theater  was  determined  to  be 
normal,  Z  scores  were  used  to  determine  the  probability  of 
the  average  monthly  spares  level  being  greater  than  the 
average  drop  expected  in  the  war  surge  Table  X  in  Chapter 
IV  summarized  the  probability  of  the  average  monthly  spares 
level  being  greater  than  the  average  drop  calculated  for  the 
theater.  The  values  ranged  from  0.217  in  USAFE  to  0.769  in 
PACAF . 

Since  the  underlying  distribution  of  the  minimums  could 
not  be  accurately  determined,  a  non  parametric  approach  was 
used  to  evaluate  the  effects  of  the  surge  given  the  monthly 
spares  level  is  near  its  minimum  level.  The  p  values  and 
their  lower  and  r  limits  are  given  in  Table  XI  in 

Chapter  IV.  The  probabilities  range  from  0.467  in  PACAF  to 
0.967  in  USAFE. 
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These  values  represent  the  probability  of  not  having 
enough  spares  to  handle  the  average  war  surge  drop  if  the 
spares  level  is  at  the  average  minimum  value.  This 
indicates  that  the  ability  of  the  theaters  to  surge  in  any 
given  month  can  be  severely  limited  by  the  system  when  at 
minimum  values.  It  shoul  i  be  noted  that  the  analysis  is  a 
simplified  worst  case  scenario.  There  are  many  additional 
factors  which  are  not  included  in  this  analysis.  In  the 
case  of  an  actual  war  in  one  theater,  additional  aircraft 
and  spares  would  be  deployed.  Aircraft  attrition  would  have 
to  be  factored  in  along  with  aircraft  being  down  for  other 
reaso! s  such  as  battle  damage  and  other  critical  systems 
down.  The  repair  pipeline  would  most  likely  be  accelerated 
and  perhaps  even  additional  spares  purchased  to  support  a 
vjar  .  The  factors  could  not  be  quantified  for  this  research 
but  should  be  considered  when  assessing  the  warfighting 
capability.  This  research  looked  at  the  situation  as  a 
woi^st  case  analysis.  This  is  done  by  assuming  that  aircraft 
are  always  available  except  for  the  ALE-47.  If  it  is  down, 
the  aircraft  is  down.  This  is  not  che  case  in  reality,  but 
was  used  in  this  study  for  two  reasons.  First,  the  program 
office  has  no  control  over  the  rest  of  the  aircraft  and  are 
only  concerned  with  the  ALF-47.  Second,  the  breakrate  of 
the  ALE-47  is  i ns i gn i f i can t  when  compared  with  the  total 
aircraft  breakrate.  If  the  total  aircraft  were  modeled,  the 
ALE-47  would  most  likely  be  in  the  noise  level  and  not 
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significantly  affect  the  aircraft’s  operational  suitability 
at  all.  Aircraft  are  constantly  down  for  a  variety  of 
reasons  and  the  ALE-47  should  not  be  significant. 

The  full  factorial  experiment  examined  the  contribution 
of  the  three  factors  to  the  average  monthly  spares  level  in 
each  theater.  In  each  theater,  the  results  were  the  same 
for  the  factors  in  terms  of  magnitude.  When  tested  for 
significance,  however,  the  results  varied  across  theaters. 
The  greatest  contribution  to  spares  level  is  the  amount  of 
spares  purchased.  The  two-level  maintenance  system  seems  to 
settle  down  to  a  steady  state  average  with  the  pipeline  full 
based  on  all  three  factors.  The  more  spares  that  were 
available  in  the  beginning  makes  the  average  higher  once 
steady  state  is  achieved.  Reliability  of  the  system  is  the 
next  most  important  contributor  because  it  generates  the 
failures.  Testability  or  false  removals  is  only  a  small 
portion  of  the  generation  of  failed  units  for  the  pipeline. 
Since  reliability  is  fairly  high,  this  factor  is  almost 
insignificant.  The  interaction  betwe-en  spares  purchased  and 
reliability  is  a  good  contributor.  The  conclusion  to  be 
drawn  from  this  is  that  ii  spares  purchase  size  is  based  on 
system  reliability,  remember  that  it  is  not  an  equal 
tradeoff  when  determining  the  purchase  amount  Another 
conclusion  is  that  if  achieving  the  warranted  reliability 
appears  not  to  be  a  problem,  the  best  way  Lo  improve  the 
operational  suitability  of  the  system  wo uld  be  to  purchase 
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more  spares . 


Spending  money  improving  testability  would 


have  far  less  effect  than  spending  it  on  spares. 

Recommendations 

There  are  two  areas  for  recommendat  i  on.s  in  this  study. 
The  first  set  of  recommendations  are  applicable  to  the  ALE- 
47  System  Program  Office.  They  apply  to  the  decision 
between  a  two-level  and  three-level  maintenance  concept  for 
the  system.  The  second  set  of  recommendations  apply  to  the 
research  itself.  These  recommend  areas  for  further  study 
using  the  model  developed  and  areas  for  improvement  in  the 
model  itself. 

ALE-47  System  Fro, gram  Office .  Using  the  operational 
requirements  and  program  information  available,  the  research 
indicates  that  the  system  is  supportable  with  a  two-level 
maintenance  concept  in  peacetime.  Spares  nay  get  very  tight 
if  a  war  surge  of  the  magnitude  stated  occur'’ .  The  best 
method  to  improve  the  operational  snitabilty  of  the  system 
is  to  purchase  additional  spares.  Improvug  reliability 
will  influence  the  operational  suitability  but,  since  the 
warranty  is  already  in  place,  it  would  not  be  prudent  to 
open  the  wa rranty  to  try  to  change  the  reliability. 
Testability  has  little  effect  on  operational  suitability  and 
should  not  be  viewed  as  a  method  to  greatly  improve  the 
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system  in  the  field 


Further  Study.  The  research  produced  several  areas 
open  to  further  study.  A  more  accurate  assessment  of  the 
effects  of  a  war  is  an  area  for  further  study.  Improvements 
to  the  model  would  have  to  be  made  to  better  characterize 
the  war  surge.  Items  which  must  be  identified  would  be  the 
numbei'  of  aircraft  moved  to  the  war  theater,  the  number  of 
spares  taken  with  them,  the  possible  acceleration  of  the 
repair  pipeline,  and  whether  new  spares  would  be  built. 

Another  area  for  model  improvement  would  be  to  make  the 
model  itself  more  user  friendly.  As  it  stands  now,  some 
simulation  experience  with  SLAM  II  is  necessary  to  modify 
the  input  parameters.  A  user-friendly  system  would  allow 
the  SLAM  code  to  be  invisible  to  the  user  and  allow  the  user 
to  tailor  the  model  to  his  specific  system.  This  would  be 
of  great  value  in  evaluating  System  Operational  Requirements 
Documents,  a  task  which  system  program  offices  are  often 
required  to  perform. 

Another  area  for  additional  study  would  be  to  develop  a 
three-level  maintenance  concept  model  using  the  same 
assumptions  and  data.  This  would  allow  a  comparison  between 
the  two  concepts  to  determine  if  any  significant  differences 
in  operational  suitability  exist.  This  wo uld  be  of  great 
value  when  making  a  decision  on  the  support  concept  to 
pursue . 
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Appendix  B:  Basic  Simulation  Model  SLAM  II  Code 


QEN.RMOTLEY.HOUR  MODEL , 7/ 1 1 /89 . 30 , , . , , ,72; 
LIMITS ,16,7. 5000 ; 

INTLC ,XX( 1 ) = . 00 1475 : BREAKRATE+FRR 
INTLC,XX(2) =1 . 5;SORTIE  LENGTH 
INTLC ,XX(3) =1 :ON  EQUIP  M  TIME 
INTLC ,XX(4) = 1 : OFF  EQUIP  M  TIME 
INTLC ,XX( 5) =. 01 ; FALSE  REMOVAL  RATE 
INTLC ,XX(6) =. 01 ; QC  FAIL  RATE 
INTLC  ,XX(7)  =0  :  A  i^OST  SORTIE  COUNTER 
INTLC.XXCS) =0;B  LOST  SORTIE  COUNTER 
INTLC ,XX(9) =0 ; C  LOST  SORTIE  COUNTER 
INTLC .XX( 10) =. 025641 ; SGR  (CONUS  &  USAFE) 
INTLC.XXdl)  =  .064  10  26;  SGR  (PACAF) 

INTLC , XX (12) = 12 ; FLYING  DAY  LENGTH 
INTLC ,XX( 13) = 12 ; NITE  LENGTH 
INTLC ,XX( 14) = 120 ; FLYING  WEEK 
INTLC. XX(15)=4a;DOWN  DAYS 

TIME  UNIT  IS  ONE  HOUR 

;  A  IS  CONUS 
;  B  IS  USAFE 
;  C  IS  PACAF 

NETWORK; 

RES0URCE/SPLYA(75) . 1 ;  A  SPARES 

RESOURCE/SPLYB (75) , 4 :  B  SPARES 

RESOURCE/SPLYC (35) . 7 ;  C  SPARES 

RESOURCE/ACA(390) , 2 ;  A  AIRCRAFT 

RESOURCE/ACB (390) , 5 ;  B  AIRCRAFT 

RESOURCE/ACC ( 156) , 8 ;  C  AIRCRAFT 

GATE/ SHIP .CLOSED ,3,6,9,10; 

GATE/DAY, CLOSED, 11,13,15; 

GATE/ WEEK .CLOSED ,12,14,16; 

;l»*»*SORTIE  GENERATION  MODULE*«»»* 

CREATE ,XX( 10) ;  GENERATE  SORTIE  RATE 
GOON, 1 ; 

ACT  ,  , NNGAT (DAY)  . EQ .  I  , ADI ; 

ACT ,  , NNGAT (DAY)  . NE  .  1  ; 
aOON,l; 

ACT , , NNGAT (WEEK) . EQ . 1 , AWl ; 

ACT ,  , NNGAT ( WEEK )  . NE  .  1  ; 

GOON, 1 ; 

ACT , , NNRSC (ACA) . LT , 1 ,LSA: 

ACT ,  , NNRSC (ACA)  .  GE .  1  ; 

ASSIGN , ATRIB ( 1)  =1  ,ATRIB(2) =UNFRM(0 ,  1)  ; 
AWAIT (2) , ACA; AIRCRAFT  AVAILABLE’ 
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ACT/ 1 ,XX(2) ; SORTIE 
aOON ,  1  : 

ACT/ 2,  ,ATRIB(2)  .LE.XX(l)  , BS 1 ; BREAK  A 
ACT/3 , , ATRIB (2) . QT . XX( 1 ) ; NO  BREAK  A 
FREE , ACA; RELEASE  AC  FOR  ANOTHER  MISSION 
TMl  TERM; 

ADI  AWAIT( 1 1/4) ,DAY.BALK(TM1) , 1 : 

ACT . , .TMl : 

AWl  AWAIT (12/4) , WEEK , BALK ( TMl ) , 1 ; 

ACT , , ,TMi : 

LSA  ASSIGN, XX(7) =XX(7) +1 .0; 

COLCT ,XX(7) .LOST  SORTIES  A; 

TERM; 

;*»«**base  supply  M0DULE»«»»* 

BSl  GOON. 2; 

ACT , 0 , , RRl ; 

ACT  .  0  .  , BHl ; 

RRl  AWAIT ( 1 ). SPLYA; WAIT  FOR  LRU 
ACT/4 .XX(3) ;R&R  PART 
FREE , ACA; 

TERM; 

BHl  AWAIT(3) .SHIP; 

ASSIGN. ATRIB (3) =UNFRM(0 . 1  )  ; 

ASSIGN , ATRIB (5) =TRI AG ( 1 92 , 240 , 7 20 )  ; 
ASSIGN, ATRIB (6) =TRIAG( 192 ,240,720)  ; 
ASSIGN, ATRIB (7) =TRI AG ( 432 , 62 4 ,960)  ; 
ACT/5 .ATRIB(5) , ,DPT;SHIP  A  TO  DEPOT 

;  2»»»^^SORTIE  GENERATION  MODULE»»»«» 

CREATE , XX ( 10) ;  GENERATE  SORTIE  RATE 
GOON , 1 ; 

ACT . , NNGAT (DAY) . EQ . 1 , AD2 ; 

ACT ,  , NNGAT (DAY)  . NE .  1  ; 

GOON . 1 ; 

ACT , , NNGAT (WEEK) . EQ . 1 . AW2 ; 

ACT , , NNGAT (WEEK) . NE . 1 ; 

GOON . 1 ; 

ACT, ,NNRSC(ACB) .LT. 1 ,LSB; 

ACT , , NNRSC ( ACB) . GE . 1 ; 

ASSIGN . ATRIB ( 1 ) =2 , ATRIB (2) =UNFRM(0 , 1 ) ; 
AWAIT(5) .ACB; AIRCRAFT  AVAILABLE? 

ACT/6 ,XX(2) ; SORTIE 
GOON , 1 ; 

ACT/ 7 , , ATRIB (2) . LE . XX ( 1 ) . BS2 ; BREAK  B 
ACT/a , , ATRIB (2) . GT . XX{ 1 ) ; NO  BREAK  B 
FREE , ACB ; RELEASE  AC  FOR  ANOTHER  MISSION 
TM2  TERM; 
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AD2  AWAIT ( 13/4) , DAY , BALK (TM2) , 1 ; 

ACT , , ,TM2 : 

AW2  AWAIT (14/4) .WEEK, BALK (TM2) , 1 : 

ACT . , ,TM2 : 

LSB  ASSIGN, xx(a) =xx(a) + 1 . 0 : 

COLCT .XX(a) .LOST  SORTIES  B; 

TERM; 

;»*»»*BASE  SUPPLY  MODULE##**# 

BS2  GOON, 2; 

ACT . 0  .  , RR2 ; 

ACT  .0  ,  ,BH2 : 

RR2  AWAIT (4) ,SPLYB:WAIT  FOR  LRU 
ACT/9,XX(3) ;R&R  PART 
FREE.ACB; 

TERM; 

BH2  AWAIT(6) .SHIP; 

ASSIGN , ATRIB (3) =UNFRM(0 , 1 )  ; 

ASSIGN, ATRIB (5) =TRIAG(192,2Sa,720) ; 
ASSIGN , ATRIB (6) =TRI AG ( 192 . 2aa . 720 ) ; 
ASSIGN, ATRIB  f 7 ) =TRI AG ( 432 , 62  4 , 960)  ; 

ACT/ 10 . ATRIB (5) , .DPT ; SHIP  B  TO  DEPOT 

;3**»*SORTIE  GENERATION  MODULE#**** 

CREATE.XX( 1 1) ;  GENERATE  SORTIE  RATE 
GOON, 1 ; 

ACT, .NNGAT(DAY) . EQ . 1 , AD3 ; 

ACT , , NNGAT (DAY) . NE . 1 ; 

GOON , 1 ; 

ACT, , NNGAT (WEEK) . EQ . 1 , AW3 ; 

ACT, , NNGAT (WEEK) . NE . 1 ; 

GOON . 1 ; 

ACT , , NNRSC (ACC) . LT . 1 . LSC ; 

ACT , , NNRSC (ACC) . GE . 1 ; 

ASSIGN , ATRIB (1)=3,ATRIB(2) =UNFRM(0 , 1 ) ; 
AWAIT  (8)  ,  ACC  ;  AIRCRAFT  AVAILABLE-? 

ACT/ 1 1 ,XX(2) ; SORTIE 
GOON, 1 ; 

ACT/12, .ATRIB(2) .LE.XX(l) ,BS3;BREAK  C 
ACT/13 , ,ATRIB(2) .GT.XX( 1) ;NO  BREAK  C 
FREE , ACC ; RELEASE  AC  FOR  ANOTHER  MISSION 
TM3  TERM; 

AD3  AWAIT( 15/4) , DAY, BALK(TM3) . 1 ; 

ACT . , .TM3 ; 

AW3  AWAIT (16/4) , WEEK , BALK (TM3 ) . 1 ; 

ACT , , ,TM3 ; 

LSC  ASSIGN,XX(9)=XX(9)+1.0; 

COLCT , XX (9) , LOST  SORTIES  C; 

TERM; 


;»*»»»BASE  SUPPLY  MODULE»»»»» 

> 

BS3  GOON, 2; 

ACT.O, .RR3; 

ACT  ,0  .  ,BH3 : 

RR3  AWAIT (7) , SPLYC ; WAIT  FOR  LRU 
ACT/ 14 . XX(3) ; R&R  PART  C 
FREE , ACC : 

TERM; 

BH3  AWAIT (9) . SHIP : 

ASSIGN , ATRIB (3) =UNFRM(0 . 1 ) • 

ASSIGN, ATRIB (5) =TRI AG ( 192 , 384 , 720 )  ; 

ASSIGN . ATRIB (6) =TR I AG (192, 384, 720)  ; 

ASSIGN , ATRIB (7) =TRIAG(432,624,960)  ; 

ACT/ 15 , ATRIB (5) , ,DPT : SHIP  C  TO  DEPOT 

;»«i*»kdepot  module***#* 

DPT  GOON,l; 

ACT/ 16 , 24 , ATRIB (3) .LE.XX(5) ,DSH;PART  OK 
ACT/ 17 , 24 , ATRIB (3) . GT . XX ( 5 ) ; PART  FAILED 
RPR  GOON; 

ACT/ 18 , ATRIB (7) ; DEPOT  REPAIR 
GOON  ; 

ASSIGN, ATRIB (4) =UNFRM(0,  1)  ; 

GOON . 1 ; 

ACT/ 19 , 24 , ATRIB (4 ) . LE. XX(6) , RPR ; FAIL  QC  CHECK 
ACT/20 ,24 ,ATRIB(4) .GT.XX(6) :PASS  QC  CHECK 
DSH  AWAIT ( 10) , SHIP : AWAIT  SHIPPING 
GOON , 1 : 

ACT/2  1  , ATRIB (6)  , ATRIB ( 1 ). EQ .  1  . S  1  ; SHIP  TIME  A 
ACT/22 , ATRIB (6) , ATRIB ( 1 ). EQ . 2 , S2 ; SHIP  TIME  B 
ACT/23 , ATRIB (6) . ATRIB ( 1 ). EQ . 3 . S3 ; SHIP  TIME  C 

51  FREE.SPLYA; 

TERM; 

52  FRFE.SPLYB; 

TERM; 

53  FREE. SPLYC; 

TERM; 

: **»*SHIPPING  MODULE****** 

CREATE ,,,.1,1; 

ACT . 120 ; 

SHP  OPEN, SHIP; 

ACT  ; 

CLOSE , SHIP ; 

ACT  ,168,, SHP ; 
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;**»»*FLYING  DAY  MODULE<nnf*» 

t 

CREATE ,,,.1.1; 

MEN  OPEN, DAY; 

ACT ,XX( 12) ; 

CLOSE.DAY; 

ACT , XX ( 13) , .MEN; 

;»»»»*FLYINa  WEEK»»*»* 

CREATE ,.,,1,1; 

WK  OPEN. WEEK; 

ACT , XX (  14)  ; 

CLOSE , WEEK ; 

ACT  ,  XX  (  15)  .  .  WK  ; 

ENDNETWORK; 

RECORD .TNOW.TTME ,  ,T, 24, 8760, 9480; 
VAR , NNRSC (SPLYA) ,A,CONUS,0,80; 

VAR , NNRSC (SPLYB) ,B,USAFE,0.80; 
VAR. NNRSC (SPLYC) , C . PAC AF . 0 . 80 ; 
INIT, 0,9480;  8760  HRS  TRANSIENT 
FIN; 
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Append i x  C ; 

Means  from  Basic 

Simulation  Model 

CONUS 

USAFE 

PACAF 

39 . 54839 

45. 3871 

20 . 12903 

44 . 03226 

45.6129 

25 . 29032 

43 . 387  1 

3  1  .  4  1935 

25 . 29032 

3 1 . 53065 

50 .45161 

16 . 58065 

39.6129 

43 . 48387 

24.51613 

40 . 70968 

38 . 25806 

22 . 67742 

47 . 64516 

46 . 80645 

17 .  16  129 

52 .  16  129 

4  1 . 80645 

25.51613 

38 . 93548 

42 . 67742 

21.45161 

50 . 48387 

48 . 41935 

22 . 1 2903 

37.51613 

4 1 . 25806 

21  .6129 

37. 51613 

33.51613 

19 . 90323 

37 . 29032 

49 . 87097 

23 

40 . 80645 

40 . 96774 

23 . 2258 1 

38 . 67742 

44 . 87097 

22  .  2258 1 

44 . 96774 

48.6129 

17  .  16  129 

40 .  12903 

45 .16129 

2  0 

47 . 03226 

o8 . 87097 

15 . 67742 

37 . 93548 

44 . 29032 

18.51613 

50 . 25806 

44  .  16  129 

20 . 77419 

45 . 58065 

37.6129 

22 . 96774 

46.45161 

45 .41935 

14 . 3871 

32 . 70968 

43 . 54839 

20 , 96774 

39 . 80645 

39 . 3871 

14 . 87097 

45 . 09677 

32 . 70968 

25 . 25806 

41 .6129 

44 . 06452 

20 . B0645 

44 . 96774 

33 . 387 1 

19 . 29032 

34.51613 

45 . 54839 

18 

34 . 387  1 

48 

2 1 . 30645 

35 . 70968 

4 1 . 25806 

22.45161 

84 


30.0  36.0  42.0  48.0  54.0 

CONUS  MEANS 


APPROX.  WILK-SHAPIRO  0.9844  30  CASES  PLOTTED 


RANK  ITS 


31.0  36 !o  41.0  46.0  51. C 


USAFE  MEANS 


APPROX.  WILK-SHAPIRO  0.9464 


30  CASES  PLOTTED 


s 


+ 


+  +  2 

2  +  2 

2  3 

+  +  +  + 

+  2 


4 . 0 


17.0  20.0  23.0 

PACAF  MEANS 


26 . 0 


APPROX.  WILK-SHAPIRO  0.9684  30  CASES  PLOTTED 


TS 


APPROX.  WILK-SHAPIRO  0.9787  30  CASES  PLOTTED 
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Appendix  E:  Theater  Histograms  for  Count 
of  Daily  Spares  Levels 


sabq  jo  aaqainjj 


aa 


lOAo'i  SP  U-ilc; 
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Append i x  F : 


Paired  t  Test  Results  for  Shippin 
Sensitivity  Analysis 


PAIRED  T  TEST  FOR  C0MUS(7)  -  C0MUS{2) 
MEAN  -3.438  STD  ERROR  a.918E-01 

T  -3.85 

DF  29 

P  0.0006 

CASES  INCLUDED  30  MISSING  CASES  0 

PAIRED  T  TEST  FOR  USAFE(7)  -  USAFE(2) 
MEAW  -i,ii6  STD  ERROR  0.996 

T  -1.12 

DF  29 

P  0.2716 

CASES  INCLUDED  30  MISSING  CASES  0 


PAIRED  T  TEST  FOR  PACAF(7)  -  PACAF(2) 
MEAN  -e.l83E-01  STD  ERROR  9.098E-01 

T  -0.68 

DF  29 

P  0.5021 

CASES  INCLUDED  30  MISSING  CASES  0 

PAIRED  T  ^EST  FOR  CONUS (7)  -  CONUS (14' 

MEAN  2.722  STD  ERROR  1.092 

T  2.49 

DF  29 

P  0.0187 

CASES  INCLUDED  30  MISSING  CASES  0 

PAIRED  T  TEST  FOR  USAFE(7)  -  USAFE(14) 
MEAN  4.827  STD  ERROR  1.107 

T  4.36 

DF  29 

P  0.0001 

CASES  INCLUDED  30  MISSING  CASES  0 

PAIRED  T  TEST  FOR  PACAF(7)  -  PACAF(14) 
MEAN  2.211  STD  ERROR  7.346E-01 

T  3.01 

DF  29 

P  0.0054 

CASES  INCLUDED  30  MISSING  CASES  0 


Time 
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UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  FOR  CONUS 


PREDICTOR 

VARIABLES 

COEFFICIENT 

STD  ERROR 

STUDENT'S  T 

CONSTANT 

39. 080 

3. 6989E-01 

105.65 

R 

17 . 577 

3.6989E-01 

47 , 52 

T 

9.4341E-01 

3.6989E-01 

2.55 

SP 

29.362 

3.6989E-01 

79.38 

RT 

-1 . 3488 

3.6989E-01 

-3.65 

RSP 

7.8585 

3 , 6989E-01 

21.25 

TSP 

1.0063 

3.6989E-01 

2 . 72 

RTSP 

-1.2859 

3.6989E-01 

-3.48 

CASES  INCLUDED  240  MISSING  CASES  0 

DEGREES  OF  FREEDOM  232 

OVERALL  F  1.293E'^03  P  VALUE  0.0000 

rtDJUSTED  R  SQUARED  0.9743 
R  SQUARED  0.9750 

RESID.  MEAN  SQUARE  32.84 


UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  FOR  USAFE 
PREDICTOR 


VARIABLES 

COEFFICIENT 

STD  ERROR 

STUDENT’S  T 

CONSTANT 

39.  122 

3.3652E-01 

116.26 

R 

18. 149 

3  3652E-01 

53 . 93 

T 

1 . 3540 

3.3652E-01 

4 . 02 

SP 

29.063 

3.3652E-01 

86.37 

RT 

-0.9573 

3  3652E-01 

-2.84 

R3F 

8.0898 

3. 3652E-01 

24 . 04 

TSP 

0.9823 

3.3652E-01 

2.92 

RTSP 

-1.3290 

3.3652E-01 

-3.95 

CASES  INCLUDED 

240 

MISSING 

CASES  0 

DEGREES  OF 

FREEDOM 

232 

OVERALL  F 

1 . 57IE+03  P  VALUE 

0 . 0000 

ADJUSTED  R 

SQUARED  0 

.9787 

R  SQUARED 

0 

.9793 

RESID.  MEAN  SQUARE 

27.  18 

P 


0.0000 
0 . 0000 
0.01  10 
0 . 0000 
0.0004- 
0 . 0000 
0 . 0069 
0.0007 


P 


0.0000 
0 . 0000 
0.0001 
0 . 0000 
0.0049 
0 . 0000 
0.0040 
0 . 0002 
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UNWEIGHTED  LEAST  SQUARES  LINEAR  REGRESSION  OF  P 


PREDICTOR 

VARIABLES 

COEFFICIENT 

STD  ERROR 

STUDENT'S  T 

P 

CONSTANT 

20.689 

2.0899E-01 

99.00 

0.0000 

R 

8. 1952 

2.0899E-01 

39.21 

0.0000 

T 

-2.4032E-01 

2.0899E-01 

-1.15 

0.2500 

SP 

15. 179 

2.0a99E-01 

72.63 

0.0000 

RT 

-3.0430E-01 

2 .0899E-01 

-1.46 

0. 1426 

RSP 

2.7266 

2.0899E-01 

13.05 

0.0000 

TSP 

-1.4247E-02 

2.0899E-01 

-0.07 

0.9025 

RTSP 

-7 .9a39E-02 

2.0899E-01 

-  0.38 

0.7033 

CASES  INCLUDED  240 

MISSING 

CASES  0 

DEGREES  OF 

FREEDOM  232 

OVERALL  F 

998.2 

P  VALUE 

0.0000 

ADJUSTED  R 

SQUARED  0.9569 

R  SQUARED 

0.9679 

RESID.  MEAN  SQUARE  10.46 
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The  purpose  of  this  study  was  to  develop  a  technique  for 
evaluating  operational  suitability  of  new  avionics  systems  with 
a  two-level  maintenance  support  system.  The  technique  was  primarily 
needed  early  in  the  design  phases  of  development  programs.  The 
focus  would  be  on  could  a  system  meet  its  operational  requirements 
as  set  down  in  program  documentation  such  as  the  Systems  Operational 
Requirements  Document  (SORD)  with  a  two-level  maintenance  concept. 

The  study  had  two  objectives:  develop  the  technique  and 
demonstrate  it  on  a  real  system.  The  AN/ALE-47  Countermeasures 
Dispenser  System  was  used  to  demonstrate  the  technique.  A  SLAM  II 
simulation  model  was  developed  which  used  operational  requirements 
and  three  system  factors  as  input  parameters.  The  system  factors 
were  reliability,  testability,  and  the  intial  spares  purchase. 

Probability  distributions  were  developed  for  the  mean  and 
minimum  monthly  theater  spares  level.  The  average  drop  in  monthly 
spares  level  due  to  the  stated  war  surge  was  determined.  The 
probability  of  having  enough  spares  to  handle  the  war  surge  drop 
was  determined.  This  allowed  quantification  of  the  operational 
suitability  of  the  system  with  the  two-level  concept. 

An  additional  analysis  determined  the  general  effects  of  each 
of  the  system  factors  on  the  monthly  theater  spares  level  through 
the  use  of  a  2^  Factorial  Design.  This  information  aids  the  user 
in  determining  which  factors  should  receive  attention  to  improve 
the  operational  suitability  of  the  system  with  a  two-level 
maintenance  concept. 
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